End of the peer revue: has the peer review process lost credibility?

Sunday 31 October, 3.45pm until 5.15pm, Lecture Theatre 1

Peer review, the system whereby scientific and scholarly work is subjected to the scrutiny of other experts in the field, has long been cherished as a guarantor of academic rigour. But today it appears to be in something of a crisis. Recent controversies have included the retraction (12 years after publication) of Andrew Wakefield’s peer-reviewed paper suggesting a connection between the MMR vaccine and autism, the ‘Climategate’ scandal involving emails between climate researchers appearing to manipulate peer review to their advantage, the assertion by eminent stem cell biologists that a clique of reviewers has been blocking the publication of quality research, the ‘Glaciergate’ incident in which the Intergovernmental Panel on Climate Change was taken to task for predicating its assertions on non-peer-reviewed ‘grey literature’, and the acceptance of a hoax paper from the ‘Centre for Research in Applied Phrenology’ (CRAP) in the peer-reviewed ‘Open Information Science Journal’.

Obviously peer review has never been infallible, but it has been trusted as the best mechanism for maintaining impartiality in research. This is now challenged. Some critics have argued peer review is merely a socially-constructed smokescreen for objectifying science, a cover for commercial and ideological interests. Others complain that while peer reviewing must depend on the collaboration of a community of experts in a given field who are committed to impartiality, inevitably subjective criteria enter the process and distort what passes for peer-reviewed science: Phil Jones, the central figure in the Climategate scandal, promised to keep two research papers out of the IPCC report. ‘I will keep them out somehow – even if we have to redefine what the peer-review literature is’. With the best will in the world the individuals who constitute a ‘community of experts’ are often asked to review and referee their friends, competitors, sometimes even their rivals. Some critics argue that one of the most disturbing threats to the integrity of the peer-review system has been the growing influence of advocacy science and the implicit pressures to provide evidence to fit in with policy objectives. In numerous areas, whether concerning climate or drugs, research has become a ‘cause’, increasingly both politicised and moralised. Conversely, the same trends have turned peer review into a ‘last word’ source of authority, silencing contestation not only over scientific facts, but over what are in fact political disputes. From this perspective, voices which lack the authority of peer review are, by definition, illegitimate. Politicians frequently wave peer-review at those who challenge their evidence-based policies on everything from behavioural economics to our individual lifestyles.

Does peer review need to be amended, or should we adjust our expectations of what peer review can achieve? Can we uphold peer review without creating the impression that if something is published in a peer-reviewed paper this automatically makes it ‘true’? Is the ideal of a pursuit of truth, free from vested interests, realisable or even desirable?

Listen to session audio:


Tracey Brown
director, Sense About Science

Fred Pearce
freelance journalist; environment consultant, New Scientist; author, The Climate Files: the battle for the truth about global warming and Peoplequake

Professor Robin Lovell-Badge
group leader, stem cell biology and developmental genetics, Francis Crick Institute

Dr Richard Smith
chair of trustees, ICDDR,B; former editor, British Medical Journal; chair, Patients Know Best

Sandy Starr
communications officer, Progress Educational Trust; webmaster, BioNews

Produced by
Sandy Starr communications officer, Progress Educational Trust; webmaster, BioNews
Recommended readings
Just the facts, please

Peer review is being co-opted into the political process, says Sandy Starr, at the cost of impartiality and independence

Sandy Starr, Times Higher Education, 27 October 2010

Has the peer review process lost credibility?

Disgruntlement over aspects of peer review is nothing new within science and academia, where the vagaries of the review process can be challenging and frustrating. But it is unusual for disputes about the process to be aired publicly and prominently, and it’s worth asking why this should happen now.

Sandy Starr, Independent, 9 October 2010

Peer-reviewed journals aren't worth the paper they're written on

'Is it in a peer-reviewed journal?' journalists are meant to ask themselves before launching into another story about rice pudding causing cancer, or chocolate prolonging life. The truth is that peer review is largely hokum.

Nigel Hawkes, Independent, 22 August 2010

We’ll only listen to you if you’ve been peer-reviewed

The decree that all future debate about The Spirit Level should take place in peer-reviewed journals highlights a new censorious dynamic.

Brendan O'Neill, spiked, 4 August 2010

Breakthroughs from the Second Tier

Peer review isn’t perfect— meet 5 high-impact papers that should have ended up in bigger journals.

The Scientist, 1 August 2010

No more plodding through

ESRC's first peer-review college aims to boost the success rates of grant applications.

Neha Popat, Times Higher Education, 4 June 2010

Richard Smith: Scrap peer review and beware of “top journals”

Rather than bolster traditional peer review at 'top journals', we should abandon prepublication review and paying excessive attention to 'top journals'. Instead, let people publish and let the world decide

Richard Smith, British Medical Journal Blogs, 22 March 2010

Journal stem cell work 'blocked'

Stem cell experts say they believe a small group of scientists is effectively vetoing high quality science from publication in journals.

Pallab Ghosh, BBC, 3 February 2010

Climate change emails between scientists reveal flaws in peer review

A close reading of the hacked emails exposes the real process of science, its jealousies and tribalism

Fred Pearce, Guardian, 3 February 2010

Perspective: 'I don't know what to believe'

Understanding peer review is key to developing informed opinions about scientific research.

Tracey Brown, Nature's Peer-to-Peer blog, 20 June 2006

Session partners

in association with