Tooth Fairy science and Fairy Tale science

“Before we try to explain something, we should be sure it actually happened.”--Ray Hyman

"Tooth Fairy science" is an expression coined by Harriet Hall, M.D., (aka the SkepDoc) to refer to doing research on a phenomenon before establishing that the phenomenon exists. Tooth Fairy science is part of a larger domain that might be called Fairy Tale science: research that aims to confirm a farfetched story believed by millions of scientifically innocent minds. Fairy Tale science uses research data to explain things that haven't been proven to have actually happened. Fairy Tale scientists mistakenly think that if they have collected data that is consistent with their hypothesis, then they have collected data that confirms their hypothesis. Tooth Fairy science seeks explanations for things before establishing that those things actually exist. For example:

You could measure how much money the Tooth Fairy leaves under the pillow, whether she leaves more cash for the first or last tooth, whether the payoff is greater if you leave the tooth in a plastic baggie versus wrapped in Kleenex. You can get all kinds of good data that is reproducible and statistically significant. Yes, you have learned something. But you haven’t learned what you think you’ve learned, because you haven’t bothered to establish whether the Tooth Fairy really exists.*

Furthermore, there may be a simpler, more plausible explanation for your data. (Most readers will not find it arduous to devise an explanation for those gifts that have replaced teeth that were placed under a pillow.)

Most people love stories about magical beings who can fly, produce material goods out of nothing, heal the sick, raise the dead, or seemingly recall a past life. Ian Stevenson, a psychiatrist and head of the department of psychiatry at the University of Virginia, spent years collecting stories from people who claimed to be reincarnated. His data is extensive and he used it to make a case for present-life calamities in terms of past-life experiences. Stevenson exemplifies the circular reasoning of many Fairy Tale scientists: he used his data to support a belief in the reality of reincarnation and he used reincarnation to explain his data.

Another area of research that is prone to Tooth Fairy science is UFOlogy. A man spots a weird pattern of clouds or lights in the sky, or scars on his leg that he hadn't noticed before, and explains them by referring to alien visitation. John Mack, for example, collected stories (his data) from patients who claimed to have been abducted by aliens. He claimed the abduction experience is identical to experiences with "star people" (extraterrestrials) reported by numerous Native American tribes and other cultures.* Mack was a psychiatrist who risked his position as a professor of psychiatry at the Harvard Medical School for his belief in the reality of alien abductions. He used his data to support his belief in the reality of alien abduction and he used alien abduction to explain his data. He did leave a rich database, however, for those inclined to do a less speculative analysis of the alien abduction experience. His use of hypnosis may have contaminated much of the data, however, by influencing the memories of his patients.

Whatever else Mack believed about his abductees, he saw their experiences as spiritual and as fitting well with his own beliefs regarding spiritual transformation and larger environmental issues. ("We need to change our materialistic, destructive ways," he said) Mack's biases towards the paranormal and the spiritual both helped and hindered his research. On the one hand, his beliefs allowed him access to those who have "experienced abduction" (whatever that might actually involve). They trusted him and revealed to him what they could of their experiences. Skeptics could not hope for such access but now have more data that can be analyzed and explained in terms of such things as sleep paralysis and hypnagogic states, or frauds and hoaxes.

Classic examples of Tooth Fairy science abound in CAM and psi studies, where the existence of such things as chi, meridians, and transfer of information are assumed. Studies explain how health is restored by moving vital energy along meridians and unblocking chi, or how selecting one picture out of four at greater than chance odds demonstrates telepathy. A grocer claims to have cured a janitor's deafness by manipulating his spine and chiropractic is born, but was the man really deaf to begin with? Was he really deaf but remained deaf?* Data is collected and many studies are done to show how spinal manipulation by affecting vital energy cures not only deafness but many other things as well. A healer claims to have cured someone's cancer by chanting and consulting a great crow in the Himalayas, but did the patient really have cancer to begin with? Studies are done and statistically significant results are achieved that show chanting and crow consultation work better than chance.

When the studies show that prayer doesn't heal or that applied kinesiology or dowsing doesn't work under controlled conditions do their advocates reject a belief in spirits or energies?  No. They know their fairies exist. That's their story and they're sticking to it no matter what the evidence. They have tons of anecdotal evidence that outweighs any scientific studies that don't confirm their beliefs. Tooth Fairy science is a magnet for those who believe that the plural of anecdotes is scientific data.

more examples

Acupuncture, therapeutic touch, and homeopathy assume the existence of chi or water memory (among other things) and ignore simpler explanations that account for their data by various placebo effects or false placebos. Some CAM defenders call these "non-specific effects" and claim that they are evidence that acupuncture or homeopathy "works."* So, if the data shows that the CAM therapy doesn't work any better than a placebo, the CAM folks claim that proves their medicine is effective! As Dr. Steven Novella has noted, this creates an alternative standard for judging the merits of CAM treatments, a standard that is "actually an elimination of the standard of care."

But perhaps the most insidious and damaging double-standard that is being advocated under the banner of CAM is a separate standard of scientific research itself. The normal rules of research that have evolved over the last few centuries are being subtly altered or discarded, with clever newspeak. It is a way for proponents to choose their evidence, rather than having the evidence decide [determine?] what works and what does not work. We saw this strategy at play with the recent acupuncture study for back pain that clearly showed acupuncture was no more effective than placebo acupuncture. Proponents (propagated by an uncritical media) turned scientific logic on its head by interpreting this result as indicating that placebo acupuncture must work also (if only we could figure out how, they unconvincingly mused). (Steven Novella)*

Dr. Novella points to The Kings Fund (a health policy non-profit in the UK) as an example of promoting an alternative standard for Fairy Tale science: they advocate counting placebo effects as evidence for CAM therapies! The Kings Fund has the chutzpah to proclaim in large letters:

New research methods needed to build evidence base on effectiveness of popular complementary therapies

They call cheating a "new research method." The whole idea of having double-blind, randomized, placebo-controlled studies is to avoid deluding ourselves. CAM folks apparently want us to delude ourselves so we can continue to believe in fairies. This is a very unhealthy, unscientific approach to research.

Among the most egregious offenders of alternative standards in CAM are the energy medicine folks, whose studies are often Tooth Fairy science. Perhaps the worst of the offenders are those researchers who speculate that whatever effects have been observed are due to "quantum physics." Of course quantum physics is real, but it is pure speculation to imagine that biological processes like healing are driven by forces at the level of electrons. Biological processes like cancer, however, can be driven by a concentration of high energy photons—by ultraviolet radiation for example—at the atomic or molecular level.

Parapsychology is notorious for studies making the psi assumption: the assumption that any significant departure from the laws of chance in a test of psychic ability is evidence that something anomalous or paranormal has occurred. In fact, parapsychologists assume the existence of mysterious processes occurring independently of the laws of physics and defying what is known about space and time, and then point to their data, usually statistical in nature, as confirmation of the existence of their Tooth Fairy, psi.