Irish Universities Rankings – a reality check

So the THES rankings are out and as has become the norm the fall in rankings of Irish universities has caused an outpouring of angst, breast beating, and general melancholia. It is described as a “Crisis” , a “disaster” and such other apocalyptic monikers. 


But what is the truth? 

Well, like all rankings the THES rankings are twofold. First they are one composite measure and anyone who thinks that the activities of any large organization can be summarised in one number should think hard. Second, the rankings are that- rankings. They measure not an absolute standard of anything but a standard relative to the moving target of all universities in the mix. Lost entirely in the discourse is that on many of the metrics Irish universities have, over the 2016-20 period, improved greatly. We take 2016 as there was a methodology change in 2015.  So what has happened since 2016? The table below shows the % changes in the scores that underly the rankings. The overall ranking is a composite of these scores weighted as per below, then ranked against the tens of thousands of universities in the sample.

UniversityTeaching
(30%)
Research
(30%)
Citations
(30%)
Industry
Income (2.5%)
International
  Orientation (7/5%)
Ranking
Change
DCU-20%0%37%-17%-6%-200
May-17%31%60%2%3%50
RCSI1%34%38%40%4%50
TCD-8%-5%-1%26%4%-63
TUD (was DIT)-2%51%59%25%4%-200
UCC-13%-3%60%-6%8%50
UCD-12%-5%4%31%7%-49
UCG-1%18%11%1%4%0
UL2%44%49%15%9%0

There are elements underneath these scores – for example Teaching comprises 5 elements – the ratio of doctoral students to staff, ratio of doctoral students to undergrads, teaching reputation (derived from a survey of tens of thousands of peers) , staff student ratio and institutional income per academic staff member. Teaching overall accounts for 30% of the composite score. 15% is down to reputation while only 4.5% is down to the staff-student ratio. So when we hear someone saying “the fall in rankings is down to not enough state funding which has resulted in the deteriorating staff student ratio” , this is almost surely wrong. Even a massive change in this metric cannot operationalise a massive change in rankings, if it is only 4.5% of the overall.

Clearly the biggest changes have been in teaching. I do not have the composition of these scores. But it is reasonable to surmise that as academic fulltime staff numbers have stalled and been supplemented with adjuncts and clinical and casual staff, and as undergraduate numbers have risen the ratios of doctoral students have suffered. This is an area where one can legitimately lay blame to the state incentives. One can also note that certain universities have presidents that have pursued a growth strategy. There is little correlation between changes in the rankings and changes in the scores. This is entirely consistent with the observation that , teaching aside, the declines where they have happened are down to other universities improving (more rapidly than Irish universities.)

Advertisements

5 thoughts on “Irish Universities Rankings – a reality check

  1. Marconatrix

    Interesting. Having been out of academia for quite a while now (so forgive my ignorance, please), I hadn’t realised quite how many higher academic institutions you had now in Ireland. Not at all bad for a relatively small nation.

    Reply
  2. Mark J

    Difficult to interpret the % changes. Unfortunately the publicly available data are based on transformations of the raw data, sometimes using a ‘blend’ of country-specific adjustments, sometimes using a purchasing power parity to compare value of investments between countries (these may also vary between years?). I think the z-scores are transformed again to make a 0-100 score for each variable. This may make the scores sensitive to the distribution of scores in each year. TLDR: without the raw data, it is difficult to interpret much from the specific values for scores in different years.
    Only thing that we do know is that institutional and national research funding feed into a number of indicators, including staff:student ratio, institutional and research income.

    Reply

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s