I wanted to write a short post just pointing out that my previous post on the "decline effect" in fact has a positive message: The effect itself is now talked about and this discussion may lead to better research and better science in the longer run.
But I got side-tracked, as I often do, in the nasty slimy roots under the surface of the clear intellectual waters. I intended to use the Sir Cyril Burt case as an example of how science corrects itself, but the first five references my Googling brought up are all over the place.
Did Burt meddle with his data? And if so, was it just an innocent problem, caused by old age, or did he create fake results on purpose? To give a short summary of the scandal:
If you read the sites thrown up by the Google machine, you find pretty much what you'd expect. Wikipedia has a whiff of something (perhaps of the he-said-she-said with no real conclusions), only noticeable after you read the other sites. Friends of Burt rise up to defend him! Utter weirdos also rise up to defend him! After a while you no longer know what to think of that famous Sir Cyril Burt affair!
Over the course of his career Burt published numerous articles and books on a host of topics ranging from psychometrics to philosophy of science to parapsychology. It is his research in behavior genetics, most notably in studying the heritability of intelligence (as measured in IQ tests) using twin studies that have created the most controversy, frequently referred to as "the Burt Affair."[13][14][15][16] Shortly after Burt died it had become known that all of his notes and records had been burnt, and he was accused of falsifying research data. The 2007 Encyclopedia Britannica noted that it is widely acknowledged that his later work was flawed and
many academics agree that data were falsified, though his earlier work is often accepted as valid.[17]
From the late 1970s it was generally accepted that "he had fabricated some of the data, though some of his earlier work remained unaffected by this revelation."[17] This was due in large part to research by Oliver Gillie (1976) and Leon Kamin (1974).[18][19] The possibility of fabrication was first brought to the attention of the scientific community when Kamin noticed that Burt's correlation coefficients of monozygotic and dizygotic twins' IQ scores were the same to three decimal places, across articles – even when new data were twice added to the sample of twins. Leslie Hearnshaw, a close friend of Burt and his official biographer, concluded after examining the criticisms that most of Burt's data from after World War II were unreliable or fraudulent.[20]
Yet I was taught it in my statistics classes as an obvious case of fraud, both because the correlation coefficients remained essentially constant (.770 or .771) after new data was twice added, and because the number of identical twins reared apart, Burt was able to find, 53 pairs, appeared far too large to be credible. Note that we are talking not just about identical twins, but identical twins not growing up in the same family.
That more recent identical twin studies find correlation coefficients in the same range is not a defense of Burt's integrity unless the new studies also analyze identical twins reared apart.
I'm writing about this because what motivates researchers is not only some desperate desire for "truth." They are motivated, as are human beings in general, by friendship, by the need for money, by prior biases, by that case where your own earlier work becomes part of how you define yourself and thus attacks against it violate you, and so on. And "truth" is an elusive concept, data can be unavailable, biased or too scanty, yet time hammers on and forces researchers to publish. Paraphrasing Rumsfeld, one does research with the biases and data one has, not the biases and data one wishes to have.
Ultimately this post is in support of the criticisms of scientific research studies. I have often read arguments that something must be true because scientists said it was so. Who are we, ordinary people, to question Science?
But scientists are ordinary people, too.