Faulty academic peer review

Biased merit review
Corrupt peer review process

Nature

Research is usually so specialized that real "peers" are either close colleagues or active rivals. The pool of well-qualified but disinterested reviewers in particular fields is inevitably limited, and the choice of reviewers often involves a conscious trade-off between genuine expertise and a known personal interest in the outcome of the review. In a highly competitive environment reviewers may be tempted to exploit their privileged position. Intellectual piracy is rumoured to be quite common. It thrives on the anonymity of referees and other forms of secrecy. "Blackballing" is way in which a reviewer can discredit a rival, by providing one very strongly negative comment which neutralizes a number of generally favourable assessments. Blackballing often turns on one particular point in a written text, which or rebuttal were allowed could be easily countered.

Certain review panels are so influential that their balanced composition becomes a sensitive semi-political issue. They are then susceptible to lobbying, factional conflict and other typical organizational pathologies. Political gameplaying, conflicts of interest and excessive secrecy are some of the outcomes which cloud the credibility of this type of peer review.

Background

Peer review is any systematic procedure for collecting the views of a number of individual experts and translating them into a collective judgement. It is used in science to judge the intellectual excellence or importance of a piece of work. Scientists regularly practise peer review to appraise projects, allocated resources, monitor performance, evaluate institutions, referee manuscripts, recognize individuals, and reward achievement. Because the scientific criteria (originality, timeliness etc.) are often extended to include considerations of technological applicability, institutional strategy, disciplinary development and other policy objectives, "merit view", as this is called, inevitably plays a larger role in the expert advice employed at higher organizational levels.

Claim

Peer review is too supportive of orthodoxy. Reviewers on the whole are unadventurous, and biased against novelty. They are suspicious of original ideas, unsympathetic to inexperienced researchers, and too respectful of established disciplinary boundaries. Peer review may be satisfactory for identifying work of low quality, but its resolving power is often inadequate to distinguish work of very high or exceptional quality. It cannot substitute for the validation process of open debate of new ideas in the scientific community, nor can it protect against deliberate deception or detect fraud or piracy in research.

Peer review procedures are no safeguard against deliberate deception and are not efficacious in detecting fraud or piracy in research systems. Its most significant limitation is that it has no political leverage. It can neither bear excessive pressure from business or sponsors, nor direct executive responsibility. If given too much power within its parent body, it is susceptible to lobbying, factional conflict and other typical organizational pathologies.

Counter-claim

Peer review is an indispensable instrument of science policy, although sometimes too much is expected of it. It should not be used, for example, to authenticate researchers, institutions or research claims. The validity of a scientific discovery is determined by its acceptance, after open critical debate, in the scientific community.