From Abracadabra to Zombies | View All
Tooth Fairy science and Fairy Tale science
“Before we try to explain something, we should be sure it actually happened.”--Ray Hyman
"Tooth Fairy science" is an expression coined by Harriet Hall, M.D., (aka the SkepDoc) to refer to doing research on a phenomenon before establishing that the phenomenon exists. Tooth Fairy science is part of a larger domain that might be called Fairy Tale science: research that aims to confirm a farfetched story believed by millions of scientifically innocent minds. Fairy Tale science uses research data to explain things that haven't been proven to have actually happened. Fairy Tale scientists mistakenly think that if they have collected data that is consistent with their hypothesis, then they have collected data that confirms their hypothesis. Tooth Fairy science seeks explanations for things before establishing that those things actually exist. For example:
You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists.*
Furthermore, there may be a simpler, more plausible explanation for your data. (Most readers will not find it arduous to devise an explanation for those gifts that have replaced teeth that were placed under a pillow.)
Most people love stories about magical beings who can fly, produce material goods out of nothing, heal the sick, raise the dead, or seemingly recall a past life. Ian Stevenson, a psychiatrist and head of the department of psychiatry at the University of Virginia, spent years collecting stories from people who claimed to be reincarnated. His data is extensive and he used it to make a case for present-life calamities in terms of past-life experiences. Stevenson exemplifies the circular reasoning of many Fairy Tale scientists: he used his data to support a belief in the reality of reincarnation and he used reincarnation to explain his data.
Another area of research that is prone to Tooth Fairy science is UFOlogy. A man spots a weird pattern of clouds or lights in the sky, or scars on his leg that he hadn't noticed before, and explains them by referring to alien visitation. John Mack, for example, collected stories (his data) from patients who claimed to have been abducted by aliens. He claimed the abduction experience is identical to experiences with "star people" (extraterrestrials) reported by numerous Native American tribes and other cultures.* Mack was a psychiatrist who risked his position as a professor of psychiatry at the Harvard Medical School for his belief in the reality of alien abductions. He used his data to support his belief in the reality of alien abduction and he used alien abduction to explain his data. He did leave a rich database, however, for those inclined to do a less speculative analysis of the alien abduction experience. His use of hypnosis may have contaminated much of the data, however, by influencing the memories of his patients.
Whatever else Mack believed about his abductees, he saw their experiences as spiritual and as fitting well with his own beliefs regarding spiritual transformation and larger environmental issues. ("We need to change our materialistic, destructive ways," he said) Mack's biases towards the paranormal and the spiritual both helped and hindered his research. On the one hand, his beliefs allowed him access to those who have "experienced abduction" (whatever that might actually involve). They trusted him and revealed to him what they could of their experiences. Skeptics could not hope for such access but now have more data that can be analyzed and explained in terms of such things as sleep paralysis and hypnagogic states, or frauds and hoaxes.
Classic examples of Tooth Fairy science abound in CAM and psi studies, where the existence of such things as chi, meridians, and transfer of information are assumed. Studies explain how health is restored by moving vital energy along meridians and unblocking chi, or how selecting one picture out of four at greater than chance odds demonstrates telepathy. A grocer claims to have cured a janitor's deafness by manipulating his spine and chiropractic is born, but was the man really deaf to begin with? Was he really deaf but remained deaf?* Data is collected and many studies are done to show how spinal manipulation by affecting vital energy cures not only deafness but many other things as well. A healer claims to have cured someone's cancer by chanting and consulting a great crow in the Himalayas, but did the patient really have cancer to begin with? Studies are done and statistically significant results are achieved that show chanting and crow consultation work better than chance.
When the studies show that prayer doesn't heal or that applied kinesiology or dowsing doesn't work under controlled conditions do their advocates reject a belief in spirits or energies? No. They know their fairies exist. That's their story and they're sticking to it no matter what the evidence. They have tons of anecdotal evidence that outweighs any scientific studies that don't confirm their beliefs. Tooth Fairy science is a magnet for those who believe that the plural of anecdotes is scientific data.
Acupuncture, therapeutic touch, and homeopathy assume the existence of chi or water memory (among other things) and ignore simpler explanations that account for their data by various placebo effects or false placebos. Some CAM defenders call these "non-specific effects" and claim that they are evidence that acupuncture or homeopathy "works."* So, if the data shows that the CAM therapy doesn't work any better than a placebo, the CAM folks claim that proves their medicine is effective! As Dr. Steven Novella has noted, this creates an alternative standard for judging the merits of CAM treatments, a standard that is "actually an elimination of the standard of care."
But perhaps the most insidious and damaging double-standard that is being advocated under the banner of CAM is a separate standard of scientific research itself. The normal rules of research that have evolved over the last few centuries are being subtly altered or discarded, with clever newspeak. It is a way for proponents to choose their evidence, rather than having the evidence decide [determine?] what works and what does not work. We saw this strategy at play with the recent acupuncture study for back pain that clearly showed acupuncture was no more effective than placebo acupuncture. Proponents (propagated by an uncritical media) turned scientific logic on its head by interpreting this result as indicating that placebo acupuncture must work also (if only we could figure out how, they unconvincingly mused). (Steven Novella)*
Dr. Novella points to The Kings Fund (a health policy non-profit in the UK) as an example of promoting an alternative standard for Fairy Tale science: they advocate counting placebo effects as evidence for CAM therapies! The Kings Fund has the chutzpah to proclaim in large letters:
New research methods needed to build evidence base on effectiveness of popular complementary therapies
They call cheating a "new research method." The whole idea of having double-blind, randomized, placebo-controlled studies is to avoid deluding ourselves. CAM folks apparently want us to delude ourselves so we can continue to believe in fairies. This is a very unhealthy, unscientific approach to research.
Among the most egregious offenders of alternative standards in CAM are the energy medicine folks, whose studies are often Tooth Fairy science. Perhaps the worst of the offenders are those researchers who speculate that whatever effects have been observed are due to "quantum physics." Of course quantum physics is real, but it is pure speculation to imagine that biological processes like healing are driven by forces at the level of electrons. Biological processes like cancer, however, can be driven by a concentration of high energy photons—by ultraviolet radiation for example—at the atomic or molecular level.
Parapsychology is notorious for studies making the psi assumption: the assumption that any significant departure from the laws of chance in a test of psychic ability is evidence that something anomalous or paranormal has occurred. In fact, parapsychologists assume the existence of mysterious processes occurring independently of the laws of physics and defying what is known about space and time, and then point to their data, usually statistical in nature, as confirmation of the existence of their Tooth Fairy, psi.
Psychologist Jim Alcock has addressed what Dr. Hall calls Tooth Fairy science in his article about not giving the null hypothesis a chance: assuming that if x happens it is evidence of psi and then when x happens claiming that they now have evidence of psi. There may be a simpler explanation. Maybe psi doesn't exist. Departure from chance expectation in an experiment may be indicative of something interesting, but it may not be psi. Maybe no information is actually transferred. The parapsychologist may as well hypothesize that Zeus is manipulating events to provide them with misleading data:
The departure from chance expectation could be due to any number of influences—a non-random ‘random generator’, various methodological flaws, or...Zeus. (I could posit that Zeus exists and likes to torment parapsychologists, and thereby gives them significant outcomes from time to time, but does not allow replication outside parapsychology. The significant outcome would provide as much support for my hypothesis that Zeus exists as it does for the Psi hypothesis that the human subject’s volition caused the results.) (Alcock 2003)
Another area of inquiry that is prone to Tooth Fairy and Fairy Tale science is so-called creation science. Anything that happens is explained by claiming that a magical being with mighty powers did it. Data is interpreted to defend stories found in the mythology of the ancient Jews. At the time the stories of this band of desert nomads were being composed, the tribes of Israel thought of Yahweh as their local god who had selected them to be the "chosen people." They created stories of Yahweh breathing life into a clay figure and making a woman from a man's rib, of talking snakes and giant boats loaded with animals escaping death at the hand of their creator and protector. Over the centuries, Christian theologians expanded on the stories and declared Yahweh the one and only Lord of the Universe, omnipotent and omniscient, who created everything out of nothing. So now we are to believe that a most powerful being created billions of galaxies with billions of stars and who knows how many planets, but chose the ancient Jews on Earth for special consideration. The creation scientists appeal to all these stories and more like them to explain the very data they use to support the veracity of the stories. One wonders, asked Carl Sagan, why Yahweh didn't reveal to his chosen people anything about how the universe really is.*
some stories are better than others
When one hears the sound of hoofbeats in New York City, think of horses, not zebras or wildebeests. It's possible that Santa Claus really does fly around the world in a reindeer-powered sleigh and deliver presents to millions of households in a single night, but there might be a simpler explanation for all those gifts appearing on Christmas morning.
Zeus delivered them, of course, with help from the fairies.
See also ad hoc hypothesis, alternative health practices, ancient astronauts, coincidence, communal reinforcement, conditioning, confirmation bias, control study, intelligent design, integrative medicine, NCCAM, Occam's razor, placebo effect, post hoc fallacy, regressive fallacy, selective thinking, self-deception, subjective validation, testimonials, Velikovsky, wishful thinking, and Zecharia Sitchin.
I have written several articles and short pieces about alternative-health related topics that are chock full of examples of Tooth Fairy and Fairy Tale science.
Review of R. Barker Bausell's Snake Oil Science: The Truth about Complementary and Alternative Medicine
books and articles
Glymour, Clark and Douglas Stalker. "Winning Through Pseudoscience," in Philosophy of Science and the Occult, edited by Patrick Grim. 2nd ed. (Albany: State University of New York Press, 1990), pp. 75-86.
Exposing quackery in medical education by R. W. Donnell Too much Tooth Fairy science has led to what Dr. Donnell calls "quackademic medicine."