Benchmarking Journals via Google Scholar

How can arts and social science faculty show their quality to be just as high as STEM? One of the things that becomes clear when you spend any time engaged in the promotions or hiring process of universities is that there is an increased drive towards metric. This has spread from the STEM area to the AHSS areas .

Another that becomes clear is that when one is looking across faculties and areas, with the best will in the world the old adage “what gets measured gets managed” is in operation, except “what gets measured gets promoted” may be a better conceptualisation. Regardless of what anyone in a meeting or a letter says, a group of disparate assessors, including perhaps people from outside the academy who are unclear of the nuances of citation and influences, they will be in my experience more swayed towards the candidates with “more better”. Lets leave aside the issue of whether or not citations mean anything in terms of impact (my view, something but not as much as we might think).

We need, us in AHSS disciplines, to be able to benchmark our work against our STEM colleagues. Decry it as we migh, but Google is the king of big data. Google scholar is fast becoming as essential a search and bibilometric tool as any of the longer established publisher led databases. Where it excels is in its breadth, and this perhaps can be a flaw. Googlebots scour the web for citations, and aggregates them. They have recently published the 2014 journal rankings (yep, just what we need…more rankings…). Its breadth can mean that it is capturing citations to grey literature and so forth but hey, its breadth means it is capturing citations in a much wider manner than otherwise. One of the saddest letters I saw was from a very well known macroeconomist who stated that only papers in journals in the Journal Citation Reports mattered. This is delusional, and dangerously so, ignoring as it does the pattern of citation and influence. Needless to say this letter was taken seriously by the STEM based members of a panel, disregarding the fact that there are many many more, proportionally, STEM journals than AHSS journals in TR citations. Such are why we need to cast our net widely. 

The graph below shows starkly the two areas. AHSS journals, all bar the top 6 business, economic and management ones, have lower h-indices. A h-index of 20 for a journal means that 20 articles in that journal have been cited at least 20 times. Here it is actually a h-5 index, so it looks at the last 5 years only.

Clearly the top STEM journals cite more and more often than the top AHSS journals. All things being equal therefore a publication or an author in these STEM journals will have a higher h index than one in AHSS journals. This doesn’t mean that they are better – it means they operate in a different context to AHSS. That’s all.

Assuming that for the moment we are concerned with simple rankings, we should therefore be able to consider benchmarking. A publication in the second ranked BusEcoMgt journal (American Economic Review) ‘=’ one in The Lancet = one in Language Learning the second ranked in humanities and arts. Instead in other words of getting hung up on the cardinality of the journal metrics we should look at the ordinarily of them.

 

Table 1 : h5 Indices of top 20 journals

All		Bus, Eco, Mgmt		Chem and Material
Nature	355	NBER Working Papers	168	Chemical Reviews	193
The New England Journal of Medicine	329	The American Economic Review	122	Journal of the American Chemical Society	190
Science	311	Review of Financial Studies	116	Chemical Society reviews	176
The Lancet	248	The Journal of Finance	107	Nano Letters	174
Cell	223	CEPR Discussion Papers	105	Advanced Materials	173
Proceedings of the National Academy of Sciences	217	Journal of Financial Economics	101	Angewandte Chemie International Edition	171
Journal of Clinical Oncology	205	The Quarterly Journal of Economics	91	ACS Nano	149
Chemical Reviews	193	IZA Discussion Papers	81	Nature Materials	146
Physical Review Letters	191	Econometrica	75	Nature Nanotechnology	136
Journal of the American Chemical Society	190	Management Information Systems Quarterly	72	Accounts of Chemical Research	127
Nature Genetics	188	Academy of Management Journal	72	Biomaterials	118
JAMA	181	Strategic Management Journal	70	The Journal of Physical Chemistry C	115
Circulation	178	Journal of Business Ethics	70	Journal of Biological Chemistry	114
Chemical Society reviews	176	Organization Science	68	Advanced Functional Materials	111
Nano Letters	174	Journal of Management	67	Energy & Environmental Science	111
Advanced Materials	173	Review of Economics and Statistics	67	Journal of Materials Chemistry	110
Angewandte Chemie International Edition	171	Journal of Banking & Finance	66	Chemical communications (Cambridge, England)	110
NBER Working Papers	168	Journal of Marketing	65	Chemistry of Materials	107
Nucleic Acids Research	164	The Review of Economic Studies	65	Journal of Hazardous Materials	106
Journal of the American College of Cardiology	162	The Economic Journal	63	Analytical Chemistry	102
Engineerign		Medical		Social Science
Nano Letters	174	The New England Journal of Medicine	329	Computers & Education	81
Advanced Materials	173	The Lancet	248	Health Affairs	77
ACS Nano	149	Cell	223	Research Policy	73
Nature Materials	146	Proceedings of the National Academy of Sciences	217	Journal of Business Ethics	70
Nature Nanotechnology	136	Journal of Clinical Oncology	205	American Journal of Public Health	69
Nature Photonics	122	Nature Genetics	188	Global Environmental Change	66
IEEE Conference on Computer Vision and Pattern Recognition, CVPR	118	JAMA	181	Social Science & Medicine	65
The Journal of Physical Chemistry C	115	Circulation	178	American Political Science Review	54
Advanced Functional Materials	111	Journal of the American College of Cardiology	162	American Journal of Political Science	53
Energy & Environmental Science	111	Blood	156	Journal of Educational Psychology	52
Journal of Materials Chemistry	110	PLoS ONE	148	Academic Medicine	52
Renewable and Sustainable Energy Reviews	108	Nature Medicine	146	Journal of Vocational Behavior	49
Chemistry of Materials	107	Nature Reviews Cancer	135	Foreign Affairs	48
Bioresource Technology	106	Neuron	135	American Sociological Review	47
Journal of Hazardous Materials	106	British Medical Journal	133	Progress in Human Geography	47
Bioinformatics	104	Cancer Research	133	Teaching and Teacher Education	47
IEEE Transactions on Pattern Analysis and Machine Intelligence	104	The Journal of Clinical Investigation	133	Health & Place	46
Journal of Power Sources	100	The Cochrane Database of Systematic Reviews	132	Land Use Policy	46
IEEE Transactions on Industrial Electronics	98	Gastroenterology	132	Review of Educational Research	45
IEEE Transactions on Information Theory	93	Nature Immunology	129	Annual Review of Sociology	45
Humanities and Arts		Life and Earth		Physics and Math
Journal of Communication	38	Nature	355	Physical Review Letters	191
Language Learning	34	Science	311	arXiv Cosmology and Extragalactic Astrophysics (astro-ph.CO)	162
Public Opinion Quarterly	32	Cell	223	The Astrophysical Journal	153
Journal of Ethnic and Migration Studies	32	Proceedings of the National Academy of Sciences	217	arXiv High Energy Physics – Phenomenology (hep-ph)	145
Synthese	32	Nature Genetics	188	arXiv Mesoscale and Nanoscale Physics (cond-mat.mes-hall)	138
The Modern Language Journal	31	Nucleic Acids Research	164	arXiv Materials Science (cond-mat.mtrl-sci)	136
Journal of Pragmatics	31	PLoS ONE	148	Physical Review D	134
Lingua	31	Neuron	135	Journal of High Energy Physics	130
System	30	Nature Reviews Molecular Cell Biology	131	arXiv Quantum Physics (quant-ph)	128
Ethnic and Racial Studies	30	Nature Reviews Genetics	129	Physical Review B	128
Applied Linguistics	29	Nature Biotechnology	129	Nature Physics	124
Philosophical Studies	28	Cell Stem Cell	121	arXiv High Energy Physics – Theory (hep-th)	124
Gender & Society	27	Molecular Cell	121	Nature Photonics	122
Language	26	Nature Cell Biology	115	Monthly Notices of the Royal Astronomical Society	121
International Journal of Communication	26	Environmental Science & Technology	115	arXiv High Energy Physics – Experiment (hep-ex)	118
Journal for the Scientific Study of Religion	26	Genes & Development	114	Applied Physics Letters	117
Language Teaching	25	Journal of Biological Chemistry	114	Astronomy & Astrophysics	112
Political Communication	25	Nature Methods	113	arXiv Superconductivity (cond-mat.supr-con)	109
Music Perception	25	Genome Research	109	Reviews of Modern Physics	107
Journalism	25	The Journal of Cell Biology	108	arXiv High Energy Astrophysical Phenomena (astro-ph.HE)	107