Matt Ridley: On Cherry-Picking And Withholding Scientific Data

  • Date: 06/01/14
  • Matt Ridley, The Times

It’s not only Tamiflu where inconvenient data goes unpublished. Try climate science and psychology too

Perhaps it should be called Tamiflugate. Yet the doubts reported by the House of Commons Public Accounts Committee last week go well beyond the possible waste of nearly half a billion pounds on a flu drug that might not be much better than paracetamol. All sorts of science are contaminated with the problem of cherry-picked data.

The Tamiflu tale is that some years ago the pharmaceutical company Roche produced evidence that persuaded the World Health Organisation that Tamiflu was effective against flu, and governments such as ours began stockpiling the drug in readiness for a pandemic. But then a Japanese scientist pointed out that most of the clinical trials on the drug had not been published. It appears that the unpublished ones generally showed less impressive results than the published ones. [...]

To illustrate how far this problem reaches, a few years ago there was a scientific scandal with remarkable similarities, in respect of the non-publishing of negative data, to the Tamiflu scandal. A relentless, independent scientific auditor in Canada named Stephen McIntyre grew suspicious of a graph being promoted by governments to portray today’s global temperatures as warming far faster than any in the past 1,400 years — the famous “hockey stick” graph. When he dug into the data behind the graph, to the fury of its authors, especially Michael Mann, he found not only problems with the data and the analysis of it but a whole directory of results labelled “CENSORED”.

This proved to contain five calculations of what the graph would have looked like without any tree-ring samples from bristlecone pine trees. None of the five graphs showed a hockey stick upturn in the late 20th century: “This shows about as vividly as one could imagine that the hockey stick is made out of bristlecone pine,” wrote Mr McIntyre drily. (The bristlecone pine was well known to have grown larger tree rings in recent years for non-climate reasons: goats tearing the bark, which regrew rapidly, and extra carbon dioxide making trees grow faster.)

Mr McIntyre later unearthed the same problem when the hockey stick graph was relaunched to overcome his critique, with Siberian larch trees instead of bristlecones. This time the lead author, Keith Briffa, of the University of East Anglia, had used only a small sample of 12 larch trees for recent years, ignoring a much larger data set of the same age from the same region. If the analysis was repeated with all the larch trees there was no hockey-stick shape to the graph. Explanations for the omission were unconvincing.

Given that these were the most prominent and recognisable graphs used to show evidence of unprecedented climate change in recent decades, and to justify unusual energy policies that hit poor people especially hard, this case of cherry-picked publication was just as potentially shocking and costly as Tamiflugate. Omission of inconvenient data is a sin in government science as well as in the private sector.

Full comment (subscription required)