From Abracadabra to Zombies | View All
experimenter effect
The experimenter effect is a term used to describe any of a number of subtle cues or signals from an experimenter that affect the performance or response of subjects in the experiment. The cues may be unconscious nonverbal cues, such as muscular tension or gestures. They may be vocal cues, such as tone of voice. Research has demonstrated that the expectations and biases of an experimenter can be communicated to experimental subjects in subtle, unintentional ways, and that these cues can significantly affect the outcome of the experiment (Rosenthal 1998).
Robert Rosenthal has found that even slight differences in instructions given to control and experimental groups can affect the outcome of an experiment. Different vocal intonations, subtle gestures, even slight changes in posture, might influence the subjects.
Double-blind experiments can reduce experimenter effects. For example, an experimenter who works for a drug company that is trying to produce a new drug for depression would be wise to have another experimenter randomize the participants into the group getting the new drug and the group getting a placebo. The subjects should not be told what group they are in; their expectations might affect the outcome. The experimenter who does the randomization should keep to himself any information regarding who is in which group and should not be the one to distribute the pills to the subjects. The one who hands out the pills and keeps records of the effects of the participants should not know what group any of the participants is in. In this way, any bias on the part of the experimenter is minimized. Only after the experiment is concluded should the members of the groups be unblinded. (There should be an exception, of course, if something bizarre began happening, such as several patients dying of heart attacks or going on manic shopping binges. In such cases, the experiment might be halted and the participants unblinded to see if the new drug might be killing people or triggering manic episodes.)
The experimenter effect may explain why many experiments can be conducted successfully only by one person or one group of persons, while others repeatedly fail in their attempts to replicate the results. Of course, there are other reasons why studies cannot be replicated. The original experimenter may have committed errors in design, controls, or calculations. Or he may have committed fraud.
One area of research that has failed to be able to consistently produce replicable results, though its advocates have been trying to do so for more than one hundred years, is parapsychology. Fraud, incompetence, and error are common charges made by skeptics against parapsychologists. However, fraud, incompetence, and error do not seem to be restricted to parapsychologists and examples of such things can be found in all of the sciences (Smith 2003: 72; Broad and Wade). Why, then, the consistent failure at replication in psi studies? Some researchers have tried to explain the inability to replicate psi experiments by claiming that the results of experiments depend on the beliefs of the experimenters. They divide psi experimenters into two groups: the psi-conducive and the psi-inhibitory. The former are those who tend to get favorable results for psi. The latter are those who consistently do not find evidence for psi. Studies on these two types of experimenters have found that psi-conducive experimenters "come across as more enthusiastic, warmer and less egoistic than do their less successful counterparts" (Smith 2003: 77). This difference in experimenter beliefs and traits, even if true, doesn't do much to justify belief in psi. What if we found out that all of the cheery researchers believed in Zeus and none of the skeptics did? Would that justify belief in Zeus? Or, as psychologist James Alcock (2003) puts it
I could posit that Zeus exists and likes to torment parapsychologists, and thereby gives them significant outcomes from time to time, but does not allow replication outside parapsychology. The significant outcomes would provide as much support for my hypothesis that Zeus exists as it does for the Psi hypothesis....(p. 43)
It may seem to some skeptics that the notion that only true believers can get positive results in psi research is an insurmountable barrier to ever establishing psi research on par with any other type of scientific endeavor. There is, however, an even more problematic proposal made by many parapsychologists: The psychic abilities of the experimenter may directly affect the psychic abilities of the subjects in the study. In 1976, Kennedy and Taddonio introduced the expression "experimenter psi effect" to refer to "unintentional psi which affects an experimental outcome in ways that are directly related to the experimenter's needs, wishes, expectancies, moods, etc." (Smith: 79).
Alcock (2003: 35) notes that the appeal to an experimenter psi effect to explain irregularities in attempts at replicating psi experiments is simply begging the question.
When there has been a failure to replicate, it is not appropriate to engage in the circularity of assigning to this failure a label (psi-experimenter effect), and then implicitly suggesting the label as its explanation. Since there is no other way of defining or identifying the psi-experimenter effect, it has no explanatory value. Using it as a possible explanation only leads to a tautology: By substituting the definition of the psi-experimenter effect, one gets: 'The failure to replicate may be a manifestation of "one researcher failing to replicate a finding that another researcher had made".' This circular reasoning excludes from the debate a possibly fruitful aspect of research, in terms of coming to understand the reasons, other than psi, that might account for the fact that different experimenters have obtained different results.
Alcock does not think it bodes well for psi as a scientific subject if only psi-conducive researchers can replicate studies.
...what a risky adventure it would be to yield to special pleading and relax the very rules of scientific methodology that help to weed out error, self-delusion and fraud in order to admit claims that violate the basic tenets of science as we know it (2003, p. 35).
Or, as Matthew D. Smith (2003) puts it: "if sceptical researchers wishing to attempt replication cannot be expected to be successful due to their a priori beliefs about psi....then parapsychology cannot be treated as a truly scientific discipline (p.82).
See also control group experiment and hypersensory perception in the SD and the Unnatural Acts blog entry for the experimenter effect.
further reading
Alcock, James. 2003. "Give the Null Hypothesis a Chance," in Psi Wars - Getting To Grips With the Paranormal. ed. James Alcock, Jean Burns and Anthony Freeman. Imprint Academic, pp. 29-50.
Broad, William and Nicholas Wade. Betrayers of the Truth: Fraud and Deceit in the Halls of Science (Century 1982). Reprint edition: Simon & Schuster 1983.
Rosenthal, Robert. 1998. "Covert Communication in Classrooms, Clinics, and Courtrooms," Eye on Psi Chi. Vol. 3, No. 1, pp. 18-22.
exorcism