So, the Research Excellence Framework of the UK is coming closer, with end December 2013 as the cutoff date. There are many many flaws with the REF but my perspective is that some measurement of research activity is better than none. And none is what we have here in Ireland.
For those unfamiliar, the REF is at base a counting exercise, where the number and (defined in oh so many contentious ways) quality of papers published by an academic over the 2008-2013 period. It asks, in a byzantine and flawed manner admittedly, for proof of impact, at least nodding towards the notion that academic papers might have audiences beyond the academy. So, its flawed but earnest. One of the features that has now begun to emerge is that the REF might be used to move people who are not research active to more teaching orientated roles. This comes as a surprise only to those who live in a deep cave, or who have never heard the management dictat “What gets measured gets managed”
Leicester has suggested this and now Swansea management school is doing the same. They suggest that people who do not achieve a certain standard over the period will be required to teach up to 18h per week. Recall – an academic in a university is tasked with teaching and research. It seems to me very reasonable that if you are not doing one you should do the other. An 18h class contact is a full weeks work when class prep and student feedback is taken into account.
The hurdle proposed is that over the 5 years people should have published 4 papers or more in journals ranked as 3* or 4* in the Association of Business Schools’ International Guide to Academic Journal Quality. Thats 3/4 of a paper per year. This is not exactly an academic matterhorn. And yes, there are of course rebates and so forth to take into account maternity and parental leave, illness, personal circumstances etc. Bear also in mind that this comes after two previous measurement exercises, so academics in the UK are well aware of the need to produce.
We could do with something similar here. Flawed as it might be, the paper by Richard Tol shone a light. There are a tail of academics in Irish business schools across all schools who are almost or completely research inactive. The situation in other schools and disciplines in social sciences and humanities is unlikely to be any different. In the sciences it is harder but still possible to exist even within a research intensive university and not publish while not carrying a larger than normal teaching load. We might well consider some form of REF-like activity for Ireland. Excellence in teaching should be rewarded ; but activity in teaching is expected of all. Why should research be any different? Why should some people in effect free ride on the research of others? Why should some people not do any publishable research at all? Why, in effect, are some allowed to skate over an integral part of their job? These are questions the implications of whose answers would be to cause severe discomfort to some. But that does not make them invalid. Lets start by looking at business schools and disciplines (including economics). Lets start gently and define as research active only those that have published 4 papers in ANY ABS ranked journal over the last 5 years. What would we find?
__________________________________________________________
For those interested, since 2008, excluding replies and editorials, I have published 38 papers. Three more are in the accepted but not in print stage. I am very active in research, but am self admittedly more quantity than quality (although I hope that the academic gods are on the side of the big battalions). Of these 5 are not rated in the ABS – they are physics and mining journals. Of the remaining 14 are 3* or 4*. Full list below
My Publications
Title |
Year |
Source title |
ABS |
A random-matrix-theory-based analysis of stocks of markets from different countries |
2008 |
Advances in Complex Systems |
NA |
The capital markets of the middle east and north african region: Situation and characteristics |
2008 |
Emerging Markets Finance and Trade |
NA |
An ever-closer union? Examining the evolution of linkages of European equity markets via minimum spanning trees |
2008 |
Physica A: Statistical Mechanics and its Applications |
NA |
Comovements in government bond markets: A minimum spanning tree analysis |
2010 |
Physica A: Statistical Mechanics and its Applications |
NA |
The macroeconomic determinants of volatility in precious metals markets |
2010 |
Resources Policy |
NA |
Efficiency in emerging markets-Evidence from the MENA region |
2008 |
Journal of International Financial Markets, Institutions and Money |
3 |
Financial integration and emerging markets capital structure |
2011 |
Journal of Banking and Finance |
4 |
Perspectives on international and corporate finance |
2012 |
Journal of Banking and Finance |
4 |
Gravity and culture in foreign portfolio investment |
2012 |
Journal of Banking and Finance |
4 |
Halloween or January? Yet another puzzle |
2008 |
International Review of Financial Analysis |
3 |
Russian equity market linkages before and after the 1998 crisis: Evidence from stochastic and regime-switching cointegration tests |
2008 |
Journal of International Money and Finance |
3 |
he Forward Exchange Rate Bias Puzzle Is Persistent: Evidence from Stochastic and Nonparametric Cointegration Tests’ |
2009 |
Financil Review |
3 |
Is Gold a safe haven or a hedge |
2010 |
Financial ReviewFinancial Review |
3 |
Dynamics of equity market integration in Europe: Impact of political economy events |
2010 |
Journal of Common Market Studies |
3 |
Investigating the determinants of banking coexceedances in Europe in the summer of 2008 |
2010 |
Journal of International Financial Markets, Institutions and Money |
3 |
Determinants of capital structure in Irish SMEs |
2010 |
Small Business Economics |
3 |
Robust global stock market interdependencies |
2011 |
International Review of Financial Analysis |
3 |
The Irish economy: Three strikes and you’re out? |
2011 |
Panoeconomicus (formerly World Development) |
3 |
Hedges and safe havens: An examination of stocks, bonds, gold, oil and exchange rates |
2013 |
International Review of Financial Analysis |
3 |
Skewness and asymmetry in futures returns and volumes |
2008 |
Applied Financial Economics |
2 |
Reassessing co-movements among G7 equity markets: Evidence from iShares |
2008 |
Applied Financial Economics |
2 |
International influences on asset markets |
2008 |
Journal of Multinational Financial Management |
2 |
Robust global mood influences in equity pricing |
2008 |
Journal of Multinational Financial Management |
2 |
The dynamics of Central European equity market comovements |
2008 |
Quarterly Review of Economics and Finance |
2 |
Shift-contagion vulnerability in the MENA stock markets |
2009 |
World Economy |
2 |
An empirical investigation of the financial growth lifecycle |
2011 |
Journal of Small Business and Enterprise Development |
2 |
Equity market integration in the Asia Pacific region: Evidence from discount factors |
2012 |
Research in International Business and Finance |
2 |
An empirical study of multiple direct international listings |
2013 |
Global Finance Journal |
2 |
The structure of gold and silver spread returns |
2013 |
Quantitative Finance |
2 |
Do U.S. macroeconomic surprises influence equity returns? An exploratory analysis of developed economies |
2013 |
Quarterly Review of Economics and Finance |
2 |
Mood and UK equity pricing |
2008 |
Applied Financial Economics Letters |
1 |
Flights and contagion-An empirical analysis of stock-bond correlations |
2009 |
Journal of Financial Stability |
1 |
Lunar seasonality in precious metal returns? |
2010 |
Applied Economics Letters |
1 |
Volatility in the gold futures market |
2010 |
Applied Economics Letters |
1 |
Does cultural distance matter in international stock market comovement? Evidence from emerging economies around the world |
2010 |
Emerging Markets Review |
1 |
The effect of gender on stock price reaction to the appointment of directors: The case of the FTSE 100 |
2011 |
Applied Economics Letters |
1 |
London or New York: Where and when does the gold price originate? |
2013 |
Applied Economics Letters |
1 |
An analysis of forward exchange rate biasedness across developed and developing country currencies: Do observed patterns persist out of sample? |
2013 |
Emerging Markets Review |
1 |
Brian M. Lucey 
Pingback: Ninth Level Ireland » Blog Archive » Ah REF! Come on, this is Ireland … not Swansea
As a colleague often says to me, one paper a year is infinitely more than none! I agree with everything you’ve said and I would include doing mickey mouse ‘teaching and learning’ research as being equivalent to no research.
thanks! . I dont have a problem with people doing pedagogical research, per se but would imagine that this is perhaps a) best left to education specialists and b) to those that are teaching focused.
I think this is a case of being careful what you wish for, given the extensive evidence for REF-induced psychosis in senior university administrators. Also another important point here is that the REF is associated with the distribution of a significant amount of QR research funding (circa 2 billion per annum in England, figure on HEFCE website) – there is no equivalent fund in Ireland to distribute (the amount of the research block grant from the HEA seeming to be like the third secret of Fatima but likely to be around €50 million across the 7 universities). So there would be no carrot in Ireland, only stick. So whilst there is no excuse for people simply not doing their jobs (unless they are on teaching only contracts) and not engaging in meaningful research (ie. not a paper in the Irish Journal of Meaningless Bollocks every 10 years), there has to be an easier way to get around this problem than recapitulating the distorting effects of the REF (doing research to publish papers so you can be entered into the REF and earn QR money for the university, instead of doing research because it is important). ,
Andrew – yes, there are lots of laws of unintended consequences here. That said, some light > no light. And that is my main concern. Although there are some metrics of output used at institutional level I think we need finer grained public data. Also, we need to not replicate the REF etc industry of ever more REF related managers and administrators.
Publication is easy but producing REF *4 papers is not.
The idea of managing researchers is not necessarily a bad idea but I would have some concerns.
The REF warps research is all sorts of ways. For instance REF return-ability plays a big role in hiring nowadays. Those not in a permanent job therefore will tend to wait to publish within one REF assessment cycle. So if they are planning on getting a permanent job in 2014 which is in the next cycle they are going to delaying publishing everything they have till next year. Is that a good thing?
I suspect it would be better for universities just to reward good researchers directly.
The best researchers do not really need management, and it only eats into their precious time anyway. Perhaps those who really need the management shouldn’t have been hired in the first place.
Sussex is hiring.
There are two major issues with the REF.
1. It is too infrequent. Reputations will be set for 2014-2020. So it is a very big deal that crowds out all other things that are important in university management. Much better to do a smaller exercise every year or, if it is bean counting anyway, every month.
2. REF imposes lexicographic preferences. There is no trade-off between quality and quantity. Only quality matters. Or rather, the quality of the journal cover. Brian’s 10 excess 3* paper do not compensate for the fact that he has 3×4* + 1×3*.
Hi Brian – Very interesting to see an Irish academic perspective on REF. I am very much in favour of some form of Research Assessment but have serious concerns over the ‘quality’ above all else focus of the REF (as Richard alludes to). I say ‘quality’ because measuring an article’s quality based on a journal’s perceived quality (and a very limited measure of journal quality at that), is of course extremely flawed. I also feel those who write mainly for practitioners, for example about entrepreneurship, family business, teaching, other professions, should not be at a disadvantage for frequently publishing in more practitioner-focused titles which often have lower ratings. Good research is good research, no matter where it is published (the converse being true also), and I feel we do need to take account of that. I would not like to see these issues replicated, but agree with your overall sentiment regarding some for of assessment.