The ESRI-Richard Tol brouhaha has been strangely ignored by many academic blogs. I guess its because we are all on holidays…
To summarize; in May 2012 Richard, who had left the ESRI under rather strained circumstances earlier this year uploaded a working paper on the welfare-work wedge, whether and under what circumstances it is “better” to be on the welfare than to work. A part of the findings were that for some cohorts there was a significant wedge.
The paper was, according to the details on IDEAS (along with SSRN one of the larger repositories of working papers) uploaded in May.
It appears to me, but maybe I misunderstood, from the various statements that staff in the ESRI may be able to upload papers without any formal checking, but then again they are seasoned academics and researchers who will not generally issue working papers that are manifestly flawed as that will rebound on their reputation.
In any case, the newspapers picked up the findings, and it was widely reported. Here we must note that it was generally reported as “ESRI report” finding something. It was not. To describe it as such is to misunderstand the nature of the research process and to disregard the clear statement on each ESRI working paper that it is not official and should not be treated as such. Why the newspapers and other media choose to so mislead is a source of concern for another day.
In any case, yesterday the ESRI took what it described as the unprecedented step of withdrawing the (unofficial) working paper, declaring it to be in effect shoddy and error prone. It is now no longer on the ESRI working paper site, but the ever reliable Michael Taft has an extensive summary of it here and the entire paper can be downloaded here. Why it felt the need to create a precedent where none existed is unclear. This is worrying for those of us concerned with academic freedom. The ESRI press release states the ostensible reason for withdrawal as
“it has emerged that the underlying analysis requires major revision and that the paper’s estimates overstate the numbers of people who would be better off on the dole than in work.”
This seems to me to fly in the face of how research works. The ESRI working paper series is like any other. Papers start as ideas, then go to drafts then to working papers and then after informal and formal critiquing at seminars and conferences may be submitted for publication where extensive peer review suggests major revisions and then perhaps eventual publication. If the ESRI are now to require all working papers to be error free then they should set up a journal. As a journal editor the only time I would countenance the withdrawal of a paper from any series or publication is if it becomes clear that there is plagiarism, fraud or unethical behavior. None are alleged here. What is clear is that the public debate spooked the ESRI.
When the ESRI next publish a working paper on any area of controversy (and as a social science research institute that is going to happen on a regular basis) we now can no longer be sure that the publication is the thoughts of the author pure and unsullied, but is instead the outcome of internal editing. Nat O’Connor notes that the whole point of working papers is to gain comments from a wide range of people prior to further work ongoing. Editing, as we now seem to see will be the norm at the ESRI, implies control, and control is manifested in a power relationship. The role of an editor or a research institute director, in my view, is not and should not be to ensure that material which might confuse the public is not published. The role should be to ensure that the abstract and press release make clear the main finding, the strengths and weaknesses of the paper, and its role as merely one stage in academic research.
A more reasoned response from the ESRI might have been to note the unofficial, working paper nature, stress that it was part of an ongoing process, and to suggest that (as is always the case in social science) we cannot be sure and must weigh countervailing evidence. What has happened now is frank censorship – and that is not a good outcome. Nor is it feasible in the wired world.
I am neither a labour or tax economist, and so have no clue about the academic quality of the work. It seems methodologically sound, but the data are weak, and that is acknowledged in the paper. Do we wait for perfect data to investigate social phenomena, or press on, carefully, with poor data and the inevitable extra hedging of our conclusions that that will bring? I do know that Tol is a most prolific researcher who well understands the nature of the research process and who is imho unlikely to have put out knowingly flawed work. Although best known as an environment.energy economist, he has published in many areas, including tax policy. The ESRI director, Frances Ruane, is also a seasoned, well regarded researcher . When the paper, as it almost certainly will, appears in a peer reviewed academic journal with the main thrust of its findings unaltered as seems to be the case, the ESRI will stand as having for publicity or politic reasons withdrawn a working paper. That mars its reputation and the reputation of Irish academia.