From Abracadabra to Zombies
Skeptimedia
Science is fine, but we're not
3 Jan 2011. Jonah Lehrer's piece in The New Yorker called "The Truth Wears Off: Is there something wrong with the scientific method" purports to show a problem in science called the "decline effect" that needs to be addressed. In a follow-up, Lehrer writes:
We know science works. But can it work better? There is too much at stake to not ask that question. Furthermore, the public funds a vast majority of basic research—it deserves to know about any problems.
True, but the problem is not with science; it's with some scientists and with those of us in the general public who either are not skeptical enough when we read news reports of scientific studies or the studies themselves or are so skeptical that we deny the worthiness of good science and become anti-scientific quacks.
There are scientists who cheat and there are scientists who do sloppy work. There are problems with peer review and replication. But even the best scientists, following the most rigorous protocols and using the most up-to-date equipment and mathematical tools make mistakes. There are areas of science nobody in his right mind disputes, e.g., evolution, the germ theory of disease, heliocentrism. Of course, there are young Earth creationists, people who think toxins not germs cause illness, and that Galileo was wrong. So what? Science works and we already know it can work better, but not by finding the illusion of a defect that causes the decline effect. Anyone who thinks quantum mechanics can be improved by finding the defect causing a decline effect isn't thinking. What is happening in one sector of the pharmaceutical research area or in the area of parapsychology shouldn't be anyone's scientific paradigm. Pointing out obvious problems like publication bias and experimenter bias isn't a scoop.
Science can work better with better equipment and with more transparency. It can work better if journals and the general public demand to know who funds the research, if goals and methods are stated before the research begins, if studies failing to show the effectiveness of a new drug would not be filed away but be published, etc. But science can also work better if we all become a bit more skeptical about the use of "statistical significance" as a gold standard. We've become too accustomed to having other people do our thinking for us. We've got to take responsibility for developing the critical thinking skills necessary to do a proper evaluation of a mass media report on a scientific study (at the very least).
When journalists or readers of mass media publications are disturbed that two studies on the effects of broccoli consumption on cancer contradict each other, the problem is not with science. The problem is with those who don't understand how science works, don't understand how to evaluate a scientific study, and don't know what questions to ask to determine whether the study is worth taking seriously. Whose fault is that?
The problem may also be with those who do small studies or studies that jump from correlations to causal claims, and who exaggerate the significance of their work. To twist and distort real problems with scientists and invent a decline effect (which is definitely something real in parapsychology but is a completely different animal from the kind of refinements that were made over time, say, in determining the charge of an electron), as if it were some new unknown force in nature is what we might call a decline effect in journalism.
Lehrer is correct that science "is an intensely human process, shaped by all of our usual talents, tendencies, and flaws."The same can be said for journalism.
update: Sharon Begley of Newsweek jumps on the bandwagon. Who but Begley would cite studies showing vitamin D supplements are usually pointless as evidence that science-based medicine is falling asleep at the wheel? It's the quacks who push the supplements. When the studies show that people without heart disease do not benefit from statins, science-based physicians will stop prescribing them. This is not evidence of bad medicine, as Begley suggests, but of good medicine.
further reading
Chapter 8 "Causal Reasoning" and chapter 9 "Science and Pseudoscience" from Becoming a Critical Thinker (pdf files)
Skeptimedia
archives
