Education is a complex matter while reducing it to simple soundbites is easy. Ignorance, in the pure sense of not knowing, abounds when it comes to higher education. Alas, ignorance creates memes that are powerful.
Tag Archives: education analysts
This is an expanded version of the “Left Field” column published in the Irish Times.Over the last couple of years there has been a growing feeling amongst all stakeholders – government (supposedly representing the people), the administration (Department of Education and HEA), and the higher education sector that there is an increased scrutiny on what exactly academics do. Nobody is quite clear where the impetus comes from, just as nobody is quite clear what it is that the answer might be.Much of the heat has come on the issues of contact hours and how much research (and of what kind) is of any “use”. Some consider that universities and indeed all third level is merely secondary shcool for adults, and that time spent in the classroom, sorry, lecture theatre, is the only thing that is worth rewarding. These usually also show breathtaking ignorance about the process of scientific investigation, with the extreme suggesting that we should only fund reserach whose results we know in advance…..Yes, that SFI grant into Academic Clairvoyance was a good idea. But, in essence its a debate about value for money. Leaving aside the fact that not every thing that has a price has value and not everything that can be valued has a price, we can and should accept that it is simply good management practice to attempt to find out how efficient and effective our work processes are and how they might be improved. Its even more useful when the inputs, money and academic resources (although seemingly not administrative) are in scarce supply.
Efficiency is a technical concept – its about the transformation of inputs into outputs. For a given level of inputs (academics) can we get more outputs (graduates, googles, whatever). Effectiveness is a little more fuzzy, about how well we achieve our goals. In the former case we have cost and technical issues which can and should be investigated. In the latter we need to have clarity and stability about these goals in order for them to be assessed. And, in most cases we cannot even begin to measure effectiveness for years, if not decades. Plus, directed research, chasing imposed metrics, simply is not the only way to do business. There is a very long tail and a very long halflife to both “good” teaching and research. James Clerk Maxwell published in 1873 some equations he had worked on in 1865 which were abstruse and remote. That they later formed the theoretical basis of the Wireless in the early 1900s was not a directed outcome. Boole’s pamphlet on mathematical logic in 1847 had to wait for the best part of a century until the information revolution to be seen as the transcendental work that it is now seen to be. Wohler synthesised urea, closing the gap between organic and inorganic chemistry, in an accidental discovery Perkins work on dyes and the subsequent growth of synthetic organic chemistry came from a nearly discarded experiment in the synthesis of quinine. Atomic clocks were designed to test relativity and now form the basis of GPS satellites There are hundreds of examples of transformative products that emerged as byblows, serdips or sheer accidents, from basic research. In a wonderfully lucid essay in 1939 Flexner noted the importance of curiosity, serendipity and generally pootling about.
How we are moving in Ireland is towards the introduction of workload models and key performance indicators. Key performance indicators however are inherently political. The EU “knowledge triangle” suggests that higher education trades off between supporting business, education and research. Like all policy triangles it is in fact a trilemma- it is difficult if not impossible to excel in all three domains. Note that this is not to say that it is impossible to be active in all three- just that. And In Ireland we have tried for that. We have two sectors which, if they were to be closely examined have historical and natural affinities with two legs- the IoT sector has historically been orientated at teaching and business support, the University sector at teaching and research. If we require an increased emphasis on research from the IoT without additional resources, or on business support from the universities without additional resources we will degrade one or other of the existing strengths. And anyhow, how would we know what was being done was good?
Part of the problem in the assessment of what we academics to do is that they seem to do lots of things. We teach we supervise we design courses we examine we sit on committees we hunt grant money we reach out to the community…. Most workload models try to fine-tune these to a faretheewell allocating points for this and that. But in reality we do one thing – communicate knowledge, be it to the academy (research), students (teaching) or to business and society. And here is where the problem is. How can we measure these activities so as to allow managers to allocate resources and to reward those who outperform norms and expectations?
We can measure business impact in a crude fashion by patents gained and monies raised and spinoffs that rival google created. This is the dominant metric now obsessing the government, seeming to treat the universities as machines for the creation of the next FaceGoogle and wondering why there hasnt been a patent that morning. J C Maxwell or Faraday or Antonie van Leeuwenhoek would probably not have been funded by SFI. We can measure teaching eficiency in a crude fashion by how many hours an academic is in class. In the Institutes of technology there is a class contact norm of typically 16h per week. That is not the case in universities, leading to the assumption that as university academics teach less they do less. Nothing alas could be further from the truth. The key difference between the modal IoT lecturer and the modal university lecturer is the expectation and requirement of research activity. And that takes time.
It is a reasonable question to ask about the impact or import of the research even if we cant realistically answer– what is not reasonable is to assert that research is in some way less difficult or less timeconsuming than teaching. Research is akin to venture capital. We need to do a lot to get a little output. Success is rare and failure the norm. Publication in toptier journals is exceedingly rare and more than one as fleeting as a shy higgs boson in a ghillie suit on a heather covered hillside. Acceptance rates (success) in decent journals or in gaining leading research grants is often less than 10%. Research grants can take literally months to complete with no guarantee of success. And doing a paper takes time. It takes typically in excess of 100 hours of work to get a finance paper to a stage where one is happy to put it out for even working paper review. And then it takes a long time to get it published. It can take between 6 months and 6 years to get a paper from initial submission to final acceptance in finance and economics and other social sciences. In fact, in finance, economics and political sciences where the papers are usually data driven this time requirement is perhaps on the low side for social science generally. In the arts and humanities there can be literal years with “no output” as monographs and books are chronovores of enormous appetite. During that time the paper is usually under review at symposia, conferences etc and further hundreds of hours are input in refining and tweaking. The total time requirements for publication of a single article in a set of leading journals in finance were estimated to average over 1600h in a 1998 study. Meanwhile for younger academics, those seeking promotion or permanency, or seeking to move to sunnier climes they require publications so this process starts in PhD times and continues with up to a dozen projects in various stages of the pipeline. None of these times are captured in crude workload measurement approaches. to somehow assume that academics in universities engaging in research are idle is to display a gross ignorance of the scientific process – eureka moments are few and far between. If genius is 99% perspiration and 1% inspiration adequacy is 99.9% perspiration and .1% inspiration. A further aspect of research for those who are research active is peer review and editing . Typically a paper gets published after a couple of anonymous referees have commented on it. Doing this referee job is time consuming – typically 4-5h reading and writing per paper. If one does 12 reviews per annum that’s 60 hours, or the best part of two working weeks. Imagine the editor of a journal receiving 300+ papers per annum, each of which has to be read through prior to sending for review and we see another hidden time cost of research. Teaching also takes time beyond the classroom. For every hour spent in the class you take 3 to prepare, review, reflect and redo the work. Even if one has a stack of slides and a good strategy its imperative to do a mock runthrough (that takes about the same time as teaching) noting as one does what is outdated, what is wrong, what doesn’t flow etc. And then one redoes the slides and delivers, finding that in the class there are new issues to be incorporated, new questions raised, issues that seemed pellucid actually as muddled as a government jobs initiative…. So a 16h class contact in the IoT doesn’t leve a lot of time in teaching term to do any research even if one were so interested. Of course, there’s always the evening and weekends but the many IoT staff who wish to be research active typically use the summer breaks to do it –Christmas and easter are usually taken up with marking essays and so forth.
By all means therefore lets look at metrics of activity. But lets recall that these metrics typically measure output, not input and therefore cant be used as such for the measurement of efficiency or still less effectiveness. That’s not to say we should not measure research output. We should. Perhaps the issue is that we are scared of what we will find. Academics are terribly resistant to being managed and by extension to being measured. But we need to accept that without open transparent measures of research we will not allow the universities to show the extent to which they are engaged with the second leg of their historic mission. Measurement of research output is a crude proxy for research activity and even more so for research excellence. But like patents and so forth it is measureable. We don’t do this in Ireland. We have experience in the UK of several generations of research measurement. Many of us have been involved in these as assessors or as units of measurement. We can and should design a model builds on these and improves upon them, that measures research output, across units and the sector. This will show that there are many who simply do not engage in research (as measured). Every academic knows of people who simply turn up, teach and disappear. This puts an unfair burden on those who do research, and it is they who should be shouting loudest for such a metric. We have some example beyond the UK. Ontario has just done a similar exercise, and there are many measures of research activity for individual disciplines in Ireland, notably business and economics. What these all show is that there are many many academics that do not come onto the research activity radar. One can only wonder what it is that they do all day. Conceivably they are engaging with business and society but most who work in the sector would smile ruefully at that idea.
Until universities and the system managers (HEA and DES) determine what the role is of universities in relation to the trilemma we cannot however begin to reward or discipline academics or the sector for resource misallocation. What gets measured gets managed. By definition a poor measurement system will deliver poor management. But we are not measuring research in any manner in Ireland. Perhaps we should start.
This is an significantly extended version of an OpEd published by myself and Charles Larkin (whose name appears missing) in the Irish Times.
The Universities (Amendment) Bill 2012 is a shutting of the stable door after the horse has bolted and the stable sold off to a developer. It is kneejerk reaction by regulators who have failed to keep time with the pace of change in modern tertiary education, with changing educational markets, or with the balance of accountability and flexibility needed to successfully confront national and international challenges. While slapping down bolshy universities may have populist appeal, we should beware of Greeks bearing gifts. In that regard, the present proposals are a transparent attempt by the civil service to take control of the sector by plugging university policy into a centralised and dirigiste Civil Service model, and to neuter both Governing Authorities and the HEA. Irish universities used, prior to the existence of the HEA, be controlled by the Dept of Education, an unhappy time for both sides.
The core issues driving these radical proposals are payments of unauthorised allowances and an alleged breach of the Employment Control Framework in respect of promotions. In reacting to them, we need to decide what we want academia to provide for the State and how universities as institutions can best serve the common good. As in all things proportionality is also worth striving for. In an environment where universities are being placed front and center in the drive for “the smart economy” we might want to consider if command and control from bureaucrats with neither empathy for nor practical experience of these institutions is a good idea. As in so much of the education sphere the government is sending mixed messages – we want a knowledge economy but cut back on science teaching at lower levels, we want a world class university system but spend less than the OECD average on tertiary education (52k over the college span versus OECD average of 57k and EU average of 62k), we want to widen educational access but end up with no effect from “free fees”, we want more international students but make the visa and immigration process distinctly unfriendly … And now we want to have an innovative and responsive sector under the control of the civil service. To be charitable, the evidence to date for the civil service to take on board change and to assimilate rapidly changing environments is poor.
Take the ECF issue. Universities employ thousands of highly qualified internationally mobile staff. When promotion and retention decisions have to be made quickly in a fast moving and often volatile environment, there are always chances of bureaucratic feathers being ruffled. Promotion, retention and appointment must be undertaken at the pace of the needs of the students and research funders and not at the pace of the bureaucrat. At this stage it is clear that employment structures in their broadest sense need to be designed to work for the next decade and not simply in response to legacy issues that have already been disposed of.
The main provisions of the Heads of Bill are to issue directions to a university if there is concern regarding “a policy decision made by the Government or the Minister in so far it relates to the remuneration or numbers of public servants employed in that university, or a collective agreement entered into by the Government or the Minister”. There is also provision for the Minister to send in the troops in the form of an “investigator” to enquire into any of these matters, regardless of whether any cause for concern has been established. This can lead to a transfer of functions away from the universities to the Minister; or even more worryingly (since this is designed to function under the rubric of the Public Service Management Act) to the civil service bureaucracy in Marlborough Street or its agents. What is to stop a functionary deciding to engage in their own “merger mania”? Worse, what is to stop a future minister deciding to swap around bits and pieces of colleges to their own shortterm political benefit? We have a long and inglorious history of pork barrelling and local politics trumping national strategy and should be leery of giving any politician power to engage in such.
Many commentators on university education view it as essentially undergraduate focused and through a dimly recalled lens of their own experiences.Part of what drives this desire for control is the thinly disguised belief that universities are really secondary schools for young adults, that academics are lazy charlatans, that most non industry applied focused research is self-indulgent faffing about, and that the facilities lie idle most of the year. None of these accusations survive the barest scrutiny and the 2010 Comptroller and Auditor General report on Irish universities states that the sector provides good value for money under difficult conditions. That value for money is seen in the education provided to record numbers of students with reducing Exchequer funding and the growing contribution to knowledge and creativity. Perversely, these actual achievements are regularly praised by Government while at the same time, the fabric of the proposed legislation seeks to undermine them.
In this respect, the Government needs to try be more aware of the delicate balance needed to manage intellectual organizations. Universities are about human capital and knowledge creation, similar to Apple and Google. In great part their capital walks out the door every evening. Ideally, the walk (or telecommute) back the following day. Few people would think it a good idea to impose the management structures of 1920s Ford on Apple, but the Government is proposing such a course of action with its universities. The dead hand of Frederick Taylor casts a much longer shadow than one might think feasible. Knowledge organizations are different and blindly applying a civil service approach to running universities will undermine tenure (making academics more vulnerable than civil servants), change the character of academic freedom (i.e. cause academics to think twice about attacking Government policies with awkward evidence) and make Ireland more unattractive to international talent, something we need now more than ever. Machine bureaucracies, which is what universities both internally and as a sector, are but one form of organizational structure – and probably the worst suited to universities.
We only need to look across the Irish Sea to see what a command and control approach to higher education policy looks like. The Minister for Education for Wales Leighton Andrews has used his powers under the Education (Reform) Act 1988 to radically reorganize the higher education landscape with institutions being faced with stark choices of merging and/or being dissolved and face crippling financial cuts if they do not bow to the will of the Minister. That is a system without tenure, without autonomy and at the beck-and-call of parish-pump politics. It is little wonder that Wales’ higher education sector suffers from poor academic output indicators.
A win-win is needed – universities need to be freed to do their job and increase student numbers and experience success and failure – that means we need to have an adult conversation about fees. It is good that the discussion has moved from being galmost “the issue that dare not speak its name” to being front and center. Fees need to be supported either by a graduate tax or a properly functioning loan market that is totally unlike that of the US where debt and costs have been allowed to explode due to a combination of bad regulation and poor cost controls within universities. In the interim, challenge university managers to lead their institutions. Managerialism is not the solution: our ongoing experiment with the HSE should be adequate proof of that. Give them the monies that the state deems an appropriate to subsidize research and education for the common good – then let them get on with their business. If the impunity of the creators of this economic crisis not being brought to book has caused a concern about responsibility then that is perfectly fine: but making people, organizations and institutions responsible is the solution. Creating a thicket of managerial requirements will just encourage lobbying, rent seeking and the creation of a sclerotic state. Worse still, it will result in more crises and more attempts to lock the stable door after the horse has bolted. Ireland will need smart people and nimble institutions to survive the next few years. The University (Amendment) Bill stifles both.
So the crisis is now four years old, with the falling into recession of Ireland in september 2008 sounding the clear warning that there were rough times ahead.Somehow or other I (and others) were winkled out of our ivory towers to explain (or not) what was going on. There are pro and con on whether and how academics should engage with the public debates (my view is we should and should do so however we feel most comfortable), but regardless of the media, the day job goes on.
So in the last four years I have it seems published 21 peer reviewed papers, uploaded 30 working papers to SSRN presented over 40 conference presentations, examined 4 phd theses as external examiner, put 5 students through as PhD, created a masters programme which has graduated 300+ graduates, taken over the editing of two journals, edited one book, started (and stopped) a spinout, taught 14 modules at undergrad and 10 at postgraduate level, run 4 conferences with over 1000 delegates, sat on three US tenure committees as (virtual) external and formed a close working relationship with the london precious metal community.
And then there is the journalism which i think is part day job – i try to write informed by recent research. In the last year alone I have filed 25,000 words with the Examiner. What have I said? See the wordle below
Irish second level school students are now in the early stages of their school-leaving examinations, with oral language and applied science examinations ongoing and the main written exams starting Wednesday 6th with English Paper 1.They take their examination in a relatively good spot , with a dip in the numbers leaving school likely to result in less onerous requirements for university entry.
In Ireland there is a simple system for the allocation of (the vast majority) of university and related places. Students are allocated points (out of 100 in effect) on up to 6 examinations. Having submitted to the CAO their order of preference for courses the system then matches. Take a course in say Science. Lets say there are 1000 applications who have stated that this is their first choice. The system takes all their grades, and ranks the students in order of points. Lets say that the university can take 200 students on this course. Then te first two hundred students in decreasing rank order of their grades will be offered places. The 201st will not. The publication of the points by the CAO is a day of joy and sorry for tens of thousands of students. The situation is in essence an auction mechanism : students submit “bids” and the system allocates a price that clears. It is simple, foolproof, not susceptible to being gamed (once the grades are in they are in) and subject to vast criticism. The Hyland Report was quite scathing in its analysis.
Leaving aside the issue of stress, and it is real if not as bad as the situation in china where students are hooked up to vitamin and amino acid drips, there are other issues. The main criticisms are twofold. At the very top there is concern that students are taking and retaking courses to ensure “perfect” scores, which are in effect needed for entry into the leading law, veterinary and medical schools (in Ireland these are mainly entered into at undergraduate level not graduate). Universities formally do not boast of the number of high points courses or the high points needed for courses but the reality is that students and parents associate high points with academic quality. In reality, as a price, the points for courses reflect the intersection of supply and demand. Courses with low numbers of places available (for example, courses that combine a language with a law/business area) will, ceteris paribus, typically clear at a higher point (have a higher price) than those with greater places. There is a suspicion in my mind at least that over the years there has been a proliferation of such courses not only in response to consumer/social demand but also to allow universities and institutes of technology to show high point courses.
A study of highpoint achieving students was undertaken by the HEA in 2007. At the bottom there are concerns that the points system can result in students gaining entry (having enough points) to third level courses where they find themselves simply unable to undertake the rigours of third level study. This is most often focused on technology courses and results in dropout levels that are worrisome The HEA publishedin 2010 a study on retention and progression and noted that the 2007 entry cohort showed a) that overall in third level there was a dropoff of 15% from entry to the subsequent year , b) that this was significantly higher in Institutes of Technology and in lower points courses generally and c) that this was highly correlated with prior educational achievement.
More generally there is a significant concern that students have crammed and relied on rote learning for the leaving certificate and cannot then easily undertake independent critical thinking at third level. While this is not unique to Ireland it is an issue that is increasingly noted by academics (Tom Begley, Brian McCraith ) , Industry representatives (US Chamber of Commerce) even students themselves (Gary Redmond ) and of course my boss (Paddy Prendergast)
These are reasonable criticisms in so far as they go but they to my mind miss the point. The issue is not the points system per se but the way we allocate points. At present we allocate almost all university places on the basis of the points system. True, there has been a change in how students are allocated to medical schools, with students now not only assessed on their leaving cert points but also having to undertake the HPAT test which tests verbal and other reasoning skills. the HPAT is not without criticism (see the criticism by Dr James Reilly, the Minister for Health and himself a medical doctor) in particular the realisation that as students gain more experience of the test they undertake cramming for and take multiple attempts at the examination. The historic situation of free fees for irish university students has resulted in the monies that would otherwise be spent by families on that being in part diverted to feepaying and cramming schools. Free fees manifestly did not result in a significant number of students from lower socioeconomic cohorts attaining university.
What then is to be done? This blogpost was in some degree prompted by criticism of the transition year. In Ireland students typically spend 6 years in second level ; the first three years, the junior cycle, culminate in a state administered examination, and the last two years , the senior cycle, in the leaving certificate. In between schools have the option of what is called a transition year, aimed at allowing schoolkids the opportunity to undertake a variety of structured tasks with the aim of maturing themselves without examination pressure. Personally, I think its a great idea but Friends of the Elderly think otherwise suggesting that we cant afford it and that we should get students to engage in community service (which I thought was a judicial non custodial sanction for lawbreaking). Community service sounds awfully like national service…
Too often we confuse learning with hard skills, the ability to do things, when in fact what many employers need are some of these and some soft skills. Soft skills are behavioural, interpersonal, skills that are recognized as being perhaps more essential and more conducive to employment. Someone with high degrees of soft skills can learn hard skills. And soft skills matter: see here the work by James Heckman , Nobel Laureate in Economics, on this issue. Maligned though it may be Fas did undertake in 2003 a study on soft skills which is instructive. Noteworthy is the response from some industry representatives that soft skills would become more important. Soft skills include those that underly creativity and flexibility and as such are key to hightech and knowledge industries. See the comments here from the Digital Hub, here from USI, here for a more general survey and here for a study of the views of employers on soft skills. Increasingly, universities are aware of and incorporating soft skills into courses, but these are expensive and complex to instill and thus we run the risk that as budgets shrink so too will this provision.
So, back to the leaving cert and points. If we want to include more soft skills in irish graduands and the workplace in general, should we not reward that? We have seen that when we give bonus or additional points for mathematics the number of students taking higher level mathematics rises. People are instinctive economists : as excelling in leaving certificate higher mathematics costs (in terms of time and knock on effect on other courses) more than other courses students will allocate their time wisely and not take it. The reintroduction of bonus points has resulted in a rebalancing with the benefits of taking the higher mathematics course now more closely aligned with the costs thereof. Why not use the market mechanism we have in the points system and introduce points for a variety of evidenced soft skills? Already there are moves to adapt the examination format to give greater emphasis on continual assessment although this is not without controversy. But why do we not consider giving points for students that show skills in art through entry to art competitions, in music through bein part of a band, in interpersonal skills through being on a sports or debating team, in creativity through writing or developing apps, etc. Why should students not gain points for being in the scouts, or a political party, or organizing a festival, for being part of a community cleanup day and so on. If we reward people for being rounded, creative, articulate members of society they will adapt to being so. So, my call is for two new leaving cert “subjects” to be introduced in terms of points, one on “community involvement” and one on “personal skill development”. Let students show evidenced achievement in these, lets crowd source the components and weights and elements, lets use the points system to generate the kind of school graduates we as a society want and need.