Peer review?

Diverse — Drokles on March 9, 2010 at 6:48 am

Hvor Karl Popper anså videnskab, som en progression imod større viden og forståelse, hvor en teori blev afløst af en bedre anså Thomas Kuhn videnskab for at være socialt betinget, hvor teorier ganske vist afløste hinanden efter kriser, men hvor tilslutningen blev båret af kollektiv opfattelse frem for en egentlig sandhed. Jo hårdere en videnskab er jo mere gælder Popper og det beviselige og jo blødere jo mere gælder Kuhn og det sociale.

De fleste kender til mistanken fra deres skoletid om at en opgave i dansk eller samfundsfag måske i en hvis udstrækning er blevet bedømt ud fra lærerens egen moralske eller politiske forståelse. Men de færreste kender den vel fra fysik og matematik. Frank Furedi skriver i Spiked Online om peer-review processens indbyggede problemer, hvor den politiske aktivisme eller blot uerkendte overbevisning kan sløre den videnskabelige lødighed.

Peer review is a system that subjects scientific and scholarly work to the scrutiny of other experts in the field. Ideally it ensures that research is only approved or published when it meets the standards of scientific rigour and its findings are sound. At its best, peer review guarantees that it is disinterested science which informs public discussion and debate. When established through peer review, the authority of science helps to clarify disputes and injects into public discussion the latest findings and research. Peer reviewing depends on a community of experts who are competent and committed to impartiality. It depends on the commitment and collaboration of scientists and scholars in a given field.

However, the individuals who constitute a ‘community of experts’ also tend to be preoccupied with their own personal position and status. Often, the colleagues they are reviewing and refereeing are their competitors and sometimes even their bitter rivals. The contradiction between working as a member of an expert community and one’s own personal interests cannot always be satisfactorily resolved.

Unfortunately, even with the best will in the world, peer reviewing is rarely an entirely disinterested process. All too often the system of peer review is infused with vested interests. As many of my colleagues in academia know, peer reviewing is frequently carried out through a kind of mates’ club, between friends and acquaintances, and all too often the question of who gets published and who gets rejected is determined by who you know and where you stand in a particular academic debate.

(…)

Second, there is the damaging influence of nepotism and professional jealousy. Academics and researchers are all too conscious of how their prestige and career opportunities can be enhanced by getting their work published in a major journal. Sometimes, reviewers regard the research they are refereeing as the work of a competitor and adopt the tactic of either delaying or preventing its publication. This is the accusation made by 14 stem cell researchers in a letter to several major journals in their field. The researchers claim that the peer-review process was corrupted by reviewers who deliberately stalled, and even prevented, the publication of new results so that they or their associates could publish the breakthrough first. They also accused the journals of not doing enough to prevent this stalling from taking place.

The third, and in recent years the most disturbing, threat to the integrity of the peer-review system has been the growing influence of advocacy science. In numerous areas, most notably in climate science, research has become a cause and is increasingly both politicised and moralised. Consequently, in climate research, peer review is sometimes looked upon as a moral project, where decisions are influenced not simply by science but by a higher cause. The scandal surrounding ‘Climategate’ is as much about the abuse of the system of peer review as it is about the rights and wrongs of the various claims made by advocacy researchers in and around the IPCC and the UEA.

I toppen af den videnskabelig konsensus tronede FNs klimapanel IPCC, som praler med at tusindvis af videnskabsmænd har bidraget til og læst korrektur på deres rapporter. Men hvad hjælper det når kun få træffer konklusionerne.

While the IPCC insists that its critics should be judged by the most rigorous standards of peer review, it has a more relaxed attitude towards its own publications. In recent weeks there have been a series of damaging revelations about how conclusions drawn by the IPCC’s 2007 report were based on speculation and anecdotes. So claims made about disappearing mountain ice were cobbled together from information drawn from a student’s dissertation and an article published in a mountaineering magazine. Other claims were based on information from newsletters, press releases and reports produced by environmentalist advocacy groups.

There is a powerful double standard at work here: the IPCC attacks its critics for relying on ‘grey literature’ – that is, non-peer-reviewed literature – and yet it has relied on anecdotes and speculation in its reports. We shouldn’t be too surprised about this double standard, because, fundamentally, the IPCC is not simply concerned with presenting the facts but with interpreting them, giving them meaning, giving them momentum.

Det seneste eksempel på det skriver Chip Knappenberger om på MasterResource (set via Watts Up With That)

Another error in the influential reports from the Intergovernmental Panel on Climate Change (IPCC) reports has been identified. This one concerns the rate of expansion of sea ice around Antarctica.

While not an issue for estimates of future sea level rise (sea ice is floating ice which does not influence sea level), a significant expansion of Antarctic sea ice runs counter to climate model projections. As the errors in the climate change “assessment” reports from the IPCC mount, its aura of scientific authority erodes, and with it, the justification for using their findings to underpin national and international efforts to regulate greenhouse gases.

(…)

Another error in the influential reports from the Intergovernmental Panel on Climate Change (IPCC) reports has been identified. This one concerns the rate of expansion of sea ice around Antarctica.

While not an issue for estimates of future sea level rise (sea ice is floating ice which does not influence sea level), a significant expansion of Antarctic sea ice runs counter to climate model projections. As the errors in the climate change “assessment” reports from the IPCC mount, its aura of scientific authority erodes, and with it, the justification for using their findings to underpin national and international efforts to regulate greenhouse gases.

(…)

Yet, for some reason, the accompanying text claims that the trend in Figure 4.4.1b is insignificant (AR4 First Order Draft, page 4-14, lines 9-10):

The Antarctic results show a slight but insignificant positive trend of 0.7 ± 0.2% per decade.

This inconsistency was brought to the IPCC Chapter 4 authors’ attention by several IPCC commenters. Commentor John Church wrote “I do not understand why this trend is insignificant – it is more than three times the quoted error estimates” and Stefan Rahmstorf wrote “How can a trend of 0.7 +/- 0.2 be ‘insignificant’? Is not 0.2 the confidence interval, so it is significantly positive?” The IPCC responded to both in the same manner “Taken into account in revised text.”

(…)

On the topic of Antarctic sea ice trends, the “consensus of scientists”—as the IPCC likes to call itself—was wrong, led astray by the extremely poor “assessment” of the scientific knowledge-base made by a very few people who were directly involved in preparing that section—people who were either being territorial in defending and promoting their own work, were being guided by higher-ups to produce a specific IPCC point-of-view, or both.

From all I have been able to find out about this so far (including enlightenment gained from the Climategate emails into how other sections of the AR4 were carefully constructed), I would rate it “extremely unlikely” (in IPCC parlance, less than 5% chance) that what transpired was dumb luck, born of the IPCC authors’ unfamiliarity with the peer-reviewed literature—the very thing they were supposed to be assessing.

I am not sure which case is the most embarrassing.

Pinligt? Foruroligende ville jeg mene. De seneste målinger fra Antarktis ser også positive ud for de, der hellere vil have istid ifølge Watts Up With That.

0 Kommentarer »

Ingen kommentarer endnu.

RSS feed for comments on this post. TrackBack URI

Kommentér indlægget...

Monokultur kører på WordPress