Sunday 31 October, 3.45pm until 5.15pm, Lecture Theatre 1
Peer review, the system whereby scientific and scholarly work is subjected to the scrutiny of other experts in the field, has long been cherished as a guarantor of academic rigour. But today it appears to be in something of a crisis. Recent controversies have included the retraction (12 years after publication) of Andrew Wakefield’s peer-reviewed paper suggesting a connection between the MMR vaccine and autism, the ‘Climategate’ scandal involving emails between climate researchers appearing to manipulate peer review to their advantage, the assertion by eminent stem cell biologists that a clique of reviewers has been blocking the publication of quality research, the ‘Glaciergate’ incident in which the Intergovernmental Panel on Climate Change was taken to task for predicating its assertions on non-peer-reviewed ‘grey literature’, and the acceptance of a hoax paper from the ‘Centre for Research in Applied Phrenology’ (CRAP) in the peer-reviewed ‘Open Information Science Journal’.
Obviously peer review has never been infallible, but it has been trusted as the best mechanism for maintaining impartiality in research. This is now challenged. Some critics have argued peer review is merely a socially-constructed smokescreen for objectifying science, a cover for commercial and ideological interests. Others complain that while peer reviewing must depend on the collaboration of a community of experts in a given field who are committed to impartiality, inevitably subjective criteria enter the process and distort what passes for peer-reviewed science: Phil Jones, the central figure in the Climategate scandal, promised to keep two research papers out of the IPCC report. ‘I will keep them out somehow – even if we have to redefine what the peer-review literature is’. With the best will in the world the individuals who constitute a ‘community of experts’ are often asked to review and referee their friends, competitors, sometimes even their rivals. Some critics argue that one of the most disturbing threats to the integrity of the peer-review system has been the growing influence of advocacy science and the implicit pressures to provide evidence to fit in with policy objectives. In numerous areas, whether concerning climate or drugs, research has become a ‘cause’, increasingly both politicised and moralised. Conversely, the same trends have turned peer review into a ‘last word’ source of authority, silencing contestation not only over scientific facts, but over what are in fact political disputes. From this perspective, voices which lack the authority of peer review are, by definition, illegitimate. Politicians frequently wave peer-review at those who challenge their evidence-based policies on everything from behavioural economics to our individual lifestyles.
Does peer review need to be amended, or should we adjust our expectations of what peer review can achieve? Can we uphold peer review without creating the impression that if something is published in a peer-reviewed paper this automatically makes it ‘true’? Is the ideal of a pursuit of truth, free from vested interests, realisable or even desirable?
Listen to session audio:
director, Sense About Science
freelance journalist; environment consultant, New Scientist; author, The Climate Files: the battle for the truth about global warming and Peoplequake
|Professor Robin Lovell-Badge|
head, stem cell biology and developmental genetics, National Institute for Medical Research
|Dr Richard Smith|
board member, Public Library of Science; former editor, British Medical Journal; author, The Trouble with Medical Journals
communications officer, Progress Educational Trust; webmaster, BioNews
Peer review is being co-opted into the political process, says Sandy Starr, at the cost of impartiality and independenceSandy Starr, Times Higher Education, 27 October 2010
Disgruntlement over aspects of peer review is nothing new within science and academia, where the vagaries of the review process can be challenging and frustrating. But it is unusual for disputes about the process to be aired publicly and prominently, and it’s worth asking why this should happen now.Sandy Starr, Independent, 9 October 2010
'Is it in a peer-reviewed journal?' journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life. The truth is that peer review is largely hokum.Nigel Hawkes, Independent, 22 August 2010
The decree that all future debate about The Spirit Level should take place in peer-reviewed journals highlights a new censorious dynamic.Brendan O'Neill, spiked, 4 August 2010
Peer review isn’t perfect— meet 5 high-impact papers that should have ended up in bigger journals.The Scientist, 1 August 2010
ESRC's first peer-review college aims to boost the success rates of grant applications.Neha Popat, Times Higher Education, 4 June 2010
Rather than bolster traditional peer review at 'top journals', we should abandon prepublication review and paying excessive attention to 'top journals'. Instead, let people publish and let the world decideRichard Smith, British Medical Journal Blogs, 22 March 2010
Stem cell experts say they believe a small group of scientists is effectively vetoing high quality science from publication in journals.Pallab Ghosh, BBC, 3 February 2010
A close reading of the hacked emails exposes the real process of science, its jealousies and tribalismFred Pearce, Guardian, 3 February 2010
Understanding peer review is key to developing informed opinions about scientific research.Tracey Brown, Nature's Peer-to-Peer blog, 20 June 2006