Tag Archives: rankings

So…just how bad are Irish Universities?

So...just how bad are Irish Universities?

So, the THES world rankings are out and there will no doubt be the usual wailing and fingerpointing. Why are no Irish universities in the top 10? the top 100? etc etc (hint…its expensive to be there….) .
Yes, there are issues in what the rankings tell us, and I will return to that. But for the moment, look at the chart here. Its a graph of the number of top 200 universities per country scaled to millions of population. If we were the same size as the larger island next door this suggests we would have approx the same number as they do. Is UK higher ed in a terminal crisis (feel free to not answer!).

The reality is that we have world class universities. We need to invest more in them. The only question is where the money comes from. But bear in mind – there are 16,000 or so universities in the world. The top 400 where we have 5 ( in addition to TCD and UCD in the top 200 we also have UCC, NUIG and NUIM) represents the top 2.5%. The top 200 is the top 1.25%.

How many of the reporters, commentators, pundits, critics and analysts who will opine on the awful state of universities are in the top 3% of their profession in the world? How many of the politicians….?

8 Comments

October 3, 2013 · 2:18 pm

Actually, Irish universities are really outstanding.

dataIrish universities are world class. Really. So says the Shanghai rankings, the ones that perhaps are most heavily weighted against the strengths of Irish unis. How do we know? Well the 2013  rankings season kicks off with the Shanghai Academic Rankings. Expect mutterings along the lines of “sure its terrible, we have no universities in the top 100 of the Shanghai rankings” and “bloody academics, useless, wastes of money, look no Irish university in the top 100, world class me eye” and so on. No doubt similar wailing will happen when the THES rankings come out, with slightly different metrics.

Lets leave aside the peculiarities of rankings. Lets leave aside the odd rankings methodology of Shanghai which is very very heavily skewed towards a particular set of sciences. Lets look at the big picture.

There are approx 12000 universities in the world  , defined as one that have some web presence. In the 2013 Shanghai data TCD comes in in the 200-300 region, UCD and UCC in the 300-400. Thats three universites in the “top” 3%. They have been there since 2011 so this is consistent. In the broader based Times Higher we have TCD at 110 and UCD at 187, two in the top 200 or top 2%.

Question : how many Irish institutions, public or private are consistently ranked by peers, customers, and input providers as being in the top 2% of their peers in the world?

Answers on a postcard please.

12 Comments

Filed under Blogpost

The state of Irish higher education

A selection of interesting quotes from the outgoing president of IFUT 

1 Comment

April 29, 2013 · 9:54 am

TCD : ‘Leiden’ the pack in Irish universities

The Leiden world university rankings are out. We’re ranked at 48, up from 63 last year, and have therefore moved into the top 50 in the world in this ranking. UCD is ranked 281, UCC 181. Which is nice…but dont get too carried away.

The Leiden methodology uses research and collaboration metrics only – no ‘reputation’ surveys and no attempt at those problematic/questionable ‘teaching’ metrics used by the THE . So, its like all of these, partial and incomplete. But in so far as it goes it is very solid  The ranking is presented based on research impact, but views by other indicators are also available. See here for the full set .

Leiden Ranking – Research impact in Europe:

We rank at 9 in Europe. UCC is at 79 in Europe. Queen’s University Belfast is at 102 in Europe. UCD appears at 121 in Europe.

Leiden Ranking – Biomedical and Health Sciences:

We’re at 36 in the world and 8 in Europe in Biomedical and Health Sciences. UCD 243 worldwide, UCC 119.

Leiden Ranking – Life and earth sciences:

TCD is at 165 in the world and 70 in Europe. UCD 243, UCC 148 worldwide

Leiden Ranking – Mathematics and computer science:

TCD is at 73 in the world and 14 in Europe, UCD 187, UCC 358

Leiden Ranking – Natural sciences and Engineering:

TCD is 48 in the world and 10 in Europe, UCD 255, UCC 241.

Leiden Ranking – Social Sciences and Humanities

TCD is 114 in the world and 31 in Europe, UCD 248

2 Comments

Filed under Blogpost

What do university rankings measure?

Making public or institutional decisions on the basis of no measures is bad. It is not however necessarily the case that making them on the basis of flawed measures is better. In the parable of the blind men and the elephant a number of blind men feel various parts of an elephant. Each can only describe the part they themselves feel, and of course the impression of the animal is scattered and incomplete. Its only when they pool their knowledge they get a better picture

Similar issues beset university rankings. They are many and varied and no one ranking (or perhaps any ranking) can be complete. A new report from the European University Association (available here ) on the rankings issue evaluates both the overall rankings concept and the main ranking systems.

Here is the Main conclusion section (my emphasis).

4. Main conclusions

1. There have been significant new developments since the publication of the first EUA Report in 2011, including the emergence of a new venture, the Universitas 21 Rankings of National Higher Education Systems, methodological changes made in a number of existing rankings and importantly a considerable diversification in the types of products offered by several rankings providers.

2. Global university rankings continue to focus principally on the research function of the university and are still not able to do justice to research carried out in the arts, humanities and the social sciences. Moreover, even bibliometric indicators still have strong biases and flaws. The limitations of rankings remain most apparent in efforts to measure teaching performance.

3. A welcome development is that the providers of the most popular global rankings have themselves started to draw attention to the biases and flaws in the data underpinning rankings and thus to the dangers of misusing rankings.

4. New multi-indicator tools for profiling, classifying or benchmarking higher education institutions offered by the rankings providers are proliferating. These increase the pressure on and the risk of overburdening universities, obliged to collect ever more data in order to maintain as high a profile as possible. The growing volume of information being gathered on universities, and the new “products” on offer also strengthen both the influence of the ranking providers and their potential impact.

5. Rankings are beginning to impact on public policy making as demonstrated by their influence in the development of immigration policies in some countries, in determining the choice of university partner institutions, or in which cases foreign qualifications are recognised. The attention paid to rankings is also reflected in discussions on university mergers in some countries.

6. A growing number of universities have started to use data compiled from rankings for the purpose of benchmarking exercises that in turn feed into institutional strategic planning.

7. Rankings are here to stay. Even if academics are aware that the results of rankings are biased and cannot satisfactorily measure institutional quality, on a more pragmatic level they also recognise that an impressive position in the rankings can be a key factor in securing additional resources, recruiting more students and attracting strong partner institutions. Therefore those universities not represented in global rankings are tempted to calculate their likely scores in order to assess their chances of entering the rankings; everyone should bear in mind that not all publication output consists of articles in journals, and many issues relevant to academic quality cannot be measured quantitatively at all.

 

The takeaway I get is one of slight unease : the rankings are here to stay, and they re self admittedly flawed and nascent metrics of even that which they purport to measure, but these rankings are being taken much more seriously than they might be on cold analysis.  The face validity is greater than the construct validity.  And yet we see UCC has been outed as having taken great pains to ensure that its ranking on one metric was improved,  a measure described in an influential online publication as “rigging the rankings?”. It is highly improbable that UCC are alone in the world in taking such measures.

2 Comments

Filed under Blogpost

Restrictive Practices in Higher Education in Ireland

The Chairman of the Higher education Authority is no stranger to controversy. His most recent interesting comment came when he commented against the “restrictive work practices” of the third level, stating “There are very restrictive HR practices imposed on our higher education institutions by the fact that they are regarded as part of the public service, not much different from a government department or a local authority.” He also complained that Irish universities were not attracting enough foreign students, which seemed to be an issue caused by a lack of  “Greater collaboration and alignment between institutions”

I emailed, on Saturday afternoon, the HEA and inquired for specifics on these restrictive work practices. My contract, and I think every other academic, states that I will, in effect, do what I am told to do by the head of school. My duties are not specified beyond engaging in teaching as directed, carrying out research and doing such administrative duties as are assigned. It would be hard to find a more open contract than one that says “do what, where, when, for how long as, and in what manner as your boss shall dictate. End of”.  I imagine Ericcson, from whence Mr Boland hails, would be ad idem with every other company in welcoming such an open-ended specification of duties.

The HEA contacted me on Monday, with a copy of the speech (HECA J Hennessy Key Note Speech 20 April 2012 (2) (1)) and then on Wednesday kindly followed up with details of the “restrictive work practices”. I was interested to read these as I was worried that my work practices were in some way restrictive… I need not have. For the most part these are legislative, HEA driven or organizational, rather than actions carried out or not by academics.

Workload management

The Higher Education Strategy calls for a comprehensive review of existing employment contracts. It looks for contracts that are transparent and deliver accountability for appropriate workload allocation models to ensure that priorities around teaching and learning, research and administration can be managed and delivered. In relation to institutes of technology, it says that contracts should specify a minimum number of hours to be delivered on an annualised basis. Currently, the contracts in institutes of technology provide the delivery of 630 hours by assistant lecturers (560 by lecturers) over 35 weeks with a norm of 18 (16) hours per week. Because of circulars restricting the length of the academic year as well as developments such as semesterisation, the 35 weeks are never delivered. Recent agreements under Croke Park have focussed on increasing the amount of delivery per week, a less optimal approach than adopting a broader concept of the academic year.

Consideration could be given to the adoption of an annualised credit-based contract based around the current 630/560 requirements. An hour of lecturing would remain equivalent to one credit under this system but credit could then be given for other academic activities such as research, supervision of PhDs, engagement with business etc. Such a flexible approach would allow Institute management to determine credits for various activities across the differing demands across teaching Levels 6-10 as well as across the differing demands in terms of research and other academic activity. Any new contract arrangements should also provide for a level of academic and other duties – administration, management, course development, promotion of the Institute, engagement with stakeholders etc. – that form part of the normal duties of a lecturer and do not attract credits. Finally, it will be important that the contracts also state clearly what is expected in terms of attendance and the entitlements of staff in relation to annual leave.

This is strange. Lets leave aside the emphasis on institutes of technology and the equating of these with the entirety of the higher education space. As I have noted, my contract doesn’t say anything about hours or whatever. It says “do your job”. The problem with overspecifing what knowledge workers will and will not do is that they are generally smart people and will easily game the system. There is a substantial body of academic and practical research on how to ensure work gets done. Universities are not machine bureaucracies, which work by the enforcement of control. They, like all adhocracies or matrix organizations, work best when coordination and control is by the adherence to professional norms. It might be best for the HEA to contemplate how they could best set these, rather than ever-incremental micromanagement. We have a first year university course that discusses these issues and I am happy to forward the notes. They are perhaps adhocracies or As we know Irish academics work approx. 50h per week on average. Over say 48 weeks (I know its shocking that public servants will take holidays…you cant get good staff these days) that amounts to 2400h per annum.  CSO data suggest that the average weekly paid hours across the economy is approx. 32h per week. That’s just over 1500h. Restrictive work practices seem to have resulted in this sector providing a premium in terms of output of some 50%. But that’s not the real issue. The real problem is the equation of hours spent in the classroom with hours worked. That such arrant nonsense could come from the head of the government body charged with the management of higher education should be a cause of huge concern. Despite the many real issues in Irish higher education we still manage to have substantial impact on the world stage with several world class universities. By all means let me work 32h instead of 50 plus….

Redundancy

While in the sector, HEIs have considerable freedom to hire staff (subject to ECF), their capacity to make staff redundant, even where there is manifestly no work for them, does not exist.  In the institutes of technology, for instance, if staff cannot be usefully deployed due to the collapse in apprenticeship, they cannot be made redundant.  Any effort to improve efficiency by reducing unnecessary duplication of programmes across the HE sector will be rendered pointless unless a capacity for targeted redundancy is provided.  The same goes for efficiencies that could arise due to mergers of HEIs.  A capacity for targeted redundancy schemes is required.

Hire staff subject to the ECF …apart from that Mrs Lincoln, how was the play…

Again we see the IoT = all fallacy. Lets leave aside the issue of whether the government body charged with overseeing higher education might better serve society by ensuring that it works to see that apprenticeships are strengthened (Germany anyone?) rather than destroying the seed corn of future such. Today apprenticeships, tomorrow…? What market demand is there for Latin, or poetry, or for philosophy or for sociology? The higher education sector is not and cannot be simply a tool for the creation of what the HEA or industry think might be employable in three or five years time.  Since at least Newman we have known this. There is an ongoing and lively debate on how to recast this ideal but that universities play more than just a training role is surely something that the HEA might acknowledge. There is also a strange sense of competition being bad. I always thought competition, for students and ideas no less than for bread rolls, ensured that the customer or person to whom the service was provided got a better outcome. The locgical conclusion of the drive to reduce choice is silod universities, where for example UCD teaches economics and say TCD philosophy while Maynooth does Sociology. The concept of students and researchers crossing what are at best arbitrary intellectual boundaries seems anathema to the HEA. It reflects a desire for monopoly provision of education – economics 101 tells us that monopolies are always inefficient, even if they are natural monopolies, which is not at all obvious for the provision of educational programs.

Contractual matters

Particular problems are created by the way in which Ireland has transposed EU employment Directives.  Under current legislation part-time employees or those on temporary contracts can too easily acquire rights akin to permanent staff, including contracts of indefinite duration (CID).  This is particularly dangerous in a situation where HEIs are forced to rely on part time and short term contract staff. It is accepted that the HEIs have a responsibility here to ensure that contractual terms are appropriate, but in the IoT sector there is a view that they are precluded from issuing the kind of contract that would avoid a CID situation, since the form of contracts has to be agreed with unions who in turn agree these with the Department of Education and Skills.  A review of the inflexibilities generated by employment legislation, followed by legislative amendment,  is urgently needed.

Again the equation of IoT’s with the entirety of higher education…..While this may well be the case the use of the phrase “too easily acquire rights” is unfortunate to say the least. This seems to be a drive towards casualization and a backdoor abolition of not just tenure but permanency. I guess in the brave new world the HEA sees the provision of higher education as a purely market driven force, where they determine the course to be offered and organizations bid to provide same with staff hired only as and when needed. Perhaps we could organize hiring fairs or maybe the old concept of An Spailpín Fánach can be reinvigorated where gangs of underemployed Python coders and French romantic poetry specialists can hang around outside universities waving their credentials? There are of course situations where contracts are a good idea. But to create a university system where this is the norm is self defeating.  Like it or not we are in a globalized market and the market for academic talent is no different. We are now in a situation where even if a post is created it is probably going to be offered at the lowest point of the scale, which is generally now below that of comparable scales in other countries. Combine this with a total lack of job security and we will find it impossible to compete, which is in the end going to result in a poorer society in every way.

Pay

At present pay is set by government and, except in the case of the Departures Framework for universities, all HEIs must comply with pay terms nationally negotiated.  Currently, the Department of Finance, through the Department Education and Skills, plays a direct role in the establishment of salary scales, terms and conditions, appointment points on the scales, numbers of staff etc.   In the past, when time has allowed, it has been usual that negotiations have four players viz DoES, DoF, unions and ‘the employers’.  At the best of times these arrangements have been unsatisfactory in that the negotiations have been centralised and agreements are centralised, consequently much time, particularly in the IoT sector, has been spent fighting cases at local level.  In recent times, as a direct result of the economic crisis agreements have been entered into without understanding the impact that these agreements have on the functioning education (see further below).

While the HEIs do not seek complete freedom in this matter, flexibility is required to enable them to manage their workforce and their performance more effectively. HE needs a much more sophisticated architecture that is linked to both the strategic needs of institutions and their evolving structures. That architecture has to have greater flexibility and with that a series of checks and balances to underpin the flexibility.   An approach which involved freedom to pay staff within bands combined with a requirement of balance between grades (as in the current ECF) would meet many of the difficulties here.

Again the IoT seems to be driving the debate. It might be useful if the HEA clarified that they are even aware that there are two higher education systems and that no more than one size fits all the same issues do not nescessarily arise in both. The issue if there is one with Irish university pay is that it has a high mean but a low variance.  It is good to see that the HEA are beginning to suggest that this be addressed. But it is limited – why not allow managers in universities to manage? Why not let them determine, within the resources available, the pay of people. There is a market for academic labor and this should be used to signal the wages.  I would much rather we paid the most productive more than the least.

General – Management Capacity to Manage New Contractual Arrangements

 It is generally agreed that managing change in the Irish public sector is challenging.  If, as proposed above, new contractual arrangements are entered into then there will be significant challenges to middle managers in Irish HEI’s to manage those changes.  In order to do this successfully will require a much strengthened approach to PMDS, the recruitment and appointment of heads of department, deans, etc who have both academic and managerial competence.  While some institutions have developed or are developing robust systems of appointment and leadership and development to ensure management competence, anecdotally the HE system seems only patchily prepared for these changes.

This is hard to argue with in one way, stating that managers should be competent to manage. But they must also be free to manage. At present there is a widespread perception that the HEA are micro managerial zealots, desirous of interference at the lowest operational levels rather than confining themselves to policy. This may be unfair but it does exist. There is nothing wrong in principle with professional managers in universities but again there is absent from this statement an acknowledgement that knowledge workers in public or private sectors require a different style of management to other workers. Such is neither good nor bad but a fact of organizational life. There is a reason that Facebook, Google etc provide beer, Ping-Pong tables and so on and its not because the flinty eyed billionaires that run them are necessarily inherently nice people, although they may well be. Its because that approach works in that organizational space.

11 Comments

Filed under Blogpost

(why) have irish universities slipped in the rankings?

So, as Ferdinand Von Prondzinski so well states, the new academic season is one that in increasingly revolving around the beat of the various rankings. And, a new batch is now out, this time from QS. Its a well regarded ranking system, and in so far as any rankings systems go, its useful as a snapshot at least.  The snapshot, as reflected in the newspapers anyhow, is one of decline (see the Irish Independent, and the Irish Times, with the Cork Examiner delivering a slightly different message) . The “raw” rankings are much as one might expect :

“TCD drops down 13 places to 65; UCD is down 20 places from 114 to 134. NUI Galway suffers the most dramatic fall, down 66 places to 298. UCC bucked the trend, up marginally from 184 to 181.” (irish times)

So why have the rankings changed so much? Lets look at them in more detail. The universities are ranked on a number of criteria, some collected by survey, some from objective metrics. Shown below are these detailed metrics. Note that a blank indicates that the university was not ranked in the top 300 on that metric in that year ( or if they were i didnt see them …) Academic reputation is measured by a survey (in which I participated) , and accounts for 40% of the weighting. Employer reputation accounts for 20%, again via  survey ; Citations per faulty member are collected from Scopus, and it accounts for 20% as does faculty-student ratio, with the residual being split between international students and international faculty.

2009 2010 2011
TCD Academic Rep 59 63 81
Employer Rep 39 63 51
Citations 244 214 220
Staff/Student 116 145 193
International Faculty 29 29 29
International Students 93 83 94
 UCD Academic
Rep
114 120 159
Employer Rep 48 69 51
Citations
Staff/Student 138 167 232
International Faculty 54 44 54
International Students 65 61 57
 UCC Academic
Rep
219 217
Employer Rep 99 112 100
Citations
Staff/Student 232 183 XXX
International Faculty 60 80 60
International Students 204 200 228
NUIG Academic
Rep
Employer Rep 144 159 228
Citations
Staff/Student 182 158 219
International Faculty 55 50 55
International Students 145 141 178
DCU Academic
Rep
Employer Rep 141 154 100
Citations
Staff/Student 208
International Faculty 74 88
International Students 131 152 171
UL Academic
Rep
Employer Rep 176 162
Citations
Staff/Student
International Faculty 99 117
International Students 300
DIT DIT DIT
DIT Academic
Rep
Employer Rep 202 222
Citations
Staff/Student 53 152 161
International Faculty 281 281
International Students 186
NUIM Academic
Rep
Employer Rep 198
Citations
Staff/Student
International Faculty 111 133 111
International Students 220

From these we can see the causes of the generalised slipping down the rankings.  Its important to note however that these are ordinal data. They are therefore not amenable to simplistic statements (indeed, nor are most data) about “X has done worse” without careful analysis. In particular,  without knowing the distribution of the data, we cannot state that going from 152 to 161 in the world is actually significant. What we can do is look at trend.

First, its nuanced. There is actually good news here, if one interprets the figures. In general irish universities are internationally staffed and have a decent reputation amongst employers.

Second, there is a spottiness in the rankings. As might be expected the leading universities, TCD, UCC and UCD, get ranked in pretty much every area. However, looking at the top 300 rankings we see that outside these we only see occasional appearances.

Third, to me this, plus the data from subject specific rankings , suggests that there is a strong case for DIT to be a university. Its performing at least as well as the “real” universities and should, IMHO, be rewarded.

Fourth, while international faculty rankings are strong, and perhaps inflate the overall rankings, one has to wonder how  this will pan out. How easily can irish universities attract and retain staff in a tanking economy and to a sector which is apparently run by oxymoronics?

Fourth, there is some commentary that the declines are down to staff student ratios. While there clearly has been a fall in these rankings, a fall sufficiently large that it is for certain a significant fall, there are also falls in the overall academic reputation.

Fifth, the raw material of academics is knowledge – the discovery and dissemination of same. One avenue of dissemination is via students, but another is via publications. In that regard the poor showing on citations is of concern. Citations act as a (poor enough) proxy of the coproduction of knowledge and its dissemination. Increasing these might be a good way to ensure a better and more rounded university sector.

Sixth, there is much laudatory comment on UCC being “irelands first five star university”. UCC is a great university. But , these stars are something to be wary of. Unlike the other metrics these stars are opt-in. In other words, they are requested from the university side.

2 Comments

Filed under Blogpost