From Abracadabra to Zombies
reader comments: ganzfeld experiments
03 Dec 2003
While Jeff Omalanz-Hood's suggestion of a tainted library of images being
responsible for the ganzfield results sounds interesting at first, we can
see with a little analysis that it can't be the reason. Suppose a whopping
75% of the images in the library are slanted to some Western theme, and
suppose that a Western receiver suggests only Western themes and the Western
judge always picks a Western image as the guess. Even with all this Western
bias going on, you should still expect a hit rate of only 25%. Of the four
images presented to the judge as choices, one of which is the correct
choice, three of them on average will have the Western theme. The judge
eliminates the non-Western image and picks randomly from the remaining three
and gets it right 1/3 of the time, but this only happens when the randomly
chosen target is also a Western image, which happens 3/4 of the time.
Multiplying 3/4 x 1/3 we get 25%. The other 1/4 of the time, the judge is
wrong. You can try other variations on this, but it always comes out 25% as
long as everything is truly random and blind. It is interesting that the
ganzfield results come out very close to 1/3 instead of the expected 1/4, as
if one of the selections could always be eliminated. But how?
Glenn Bradford
03 Dec 2003
I want to alert Mr. Omalanz-Hood and any of your other interested readers
that the criticism he raised about content-related response bias has been
sufficiently addressed by Daryl Bem in his "Response to Hyman"
(Psychological Bulletin, 115, 22-27). Here is the relevant portion of the
article for you to reprint: Content-Related Response Bias
"Because the adequacy of target randomization cannot be statistically assessed owing to the low expected frequencies, the possibility remains open that an unequal distribution of targets could interact with receivers' content preferences to produce artifactually [?] high hit rates. As we reported in our article, Honorton and I encountered this problem in an autoganzfeld study that used a single judging set for all sessions (Study 302), a problem we dealt with in two ways. To respond to Hyman's concerns, I have now performed the same two analyses on the remainder of the database. Both treat the four-clip judging set as the unit of analysis and neither requires the assumption that the null baseline is fixed at 25% or at any other particular value. In the first analysis, the actual target frequencies observed are used in conjunction with receivers' actual judgments to derive a new, empirical baseline for each judging set. In particular, I multiplied the proportion of times each clip in a set was the target by the proportion of times that a receiver rated it as the target. This product represents the probability that a receiver would score a hit on that target if there were no psi effect. The sum of these products across the four clips in the set thus constitutes the empirical null baseline for that set. Next, I computed Cohen's measure of effect size (h) on the difference between the overall hit rate observed within that set and this empirical baseline. For purposes of comparison, I then reconverted Cohen's h back to its equivalent hit rate for a uniformly distributed judging set, in which the null baseline would, in fact, be 25%. Across the 40 sets, the mean unadjusted hit rate was 31.5%, significantly higher than 25%, one-sample t(39) = 2.44, p = .01, one-tailed. The new, bias-adjusted hit rate was virtually identical (30.7%), t(39) = 2.37, p = .01, tdiff (39) = 0.85, p = .40, indicating that unequal target frequencies were not significantly inflating the hit rate. The second analysis treats each film clip as its own control by comparing the proportion of times it was rated as the target when it actually was the target and the proportion of times it was rated as the target when it was one of the decoys. This procedure automatically cancels out any content-related target preferences that receivers (or experimenters) might have. First, I calculated these two proportions for every clip and then averaged them across the four clips within each judging set. The results show that across the 40 judging sets, clips were rated as targets significantly more frequently when they were targets than when they were decoys: 29% vs. 22%, paired t(39) = 2.03, p = .025, one-tailed. Both of these analyses indicate that the observed psi effect cannot be attributed to the conjunction of unequal target distributions and content-related response biases."
You can find the entire article on Daryl Bem's homepage at http://homepage.mac.com/dbem/homepage.html.
Jason Ewing
1 Dec 2003
I thought Jeff Omalanz-Hood's recent comments on ganzfeld experiments were
fascinating. A few years back I was berated in Skeptic for suggesting
similar possibilities by a prominent individual I won't name. At any rate, I
think Jeff's points go some way towards bringing into relief the potentially
flawed frequentistic [?] assumptions most parapsychologists seem to make.
It's unfortunate that very little skeptical analysis has been done with
regard to that. Some of the better papers/articles I've come across that
take such an approach include William
Jefferys, "Bayesian Analysis of Random Generator Data",
Robert Matthew, "Significance Levels for the Assessment of Anomalous
Phenomena", and Sidney Gendin, "ESP: A Conceptual Analysis",
Skeptical Inquirer 5, no.4 (Summer 1981).
Ross La Haye
19 Nov 2003
I was very interested to read your correspondence with Mr. Ewing about
Ganzfield experiments. As I was reading the article it occurred to me that
the test would be greatly influenced by the selection of the initial
collection of possible images to be randomized. For example, let's say you
gathered a huge library of 10,000 possible images. How would you make sure
that the library was not unconsciously slanted toward a theme? Even if the
collection were representative of all images existing, it would still be
likely that Western humans would be more likely to create images of certain
general types, perhaps of people or of places, and less likely to create
images of inanimate objects or such (this is an example--the bias could be
the other way). The difference might be small, say 8.2 %. The same Western
human prejudice would belong to at least some test subjects when describing
images "seen," and therefore some would be more likely than chance to
describe scenes that could be interpreted as like the right target, even if
the specific images presented were randomly selected from the necessarily
tainted library of choices and no sensory leakage occurred. Even if you
tried to compensate for this possibility, it would be extremely difficult to
screen each person's unconscious prejudices from hidden patterns of
preference in the initial library of images.
This possible explanation seems even more likely when you read that the best hits are from film clips, which are much more likely to be action oriented and therefore more tilted to a certain type of image and away from others.
Just a thought.
Jeff Omalanz-Hood
reply: And a good thought it is, Jeff. It has been raised before as a possible (naturalistic) explanation of the statistical anomaly produced by the Ganzfeld tests.
12 Nov 2003
I am contacting you in the hope that I can make you more aware (if only for
a moment) of your appalling and quite frankly embarrassing ignorance, of not
only the ganzfeld experiments, but grade school science.
After reporting that the results of the ganzfeld meta-analysis were at 33.2% instead of the 25% expected by chance, you try to account for the 8.2% discrepancy in a way that is truly baffling.
You write: "This sounds impressive until you examine the claim ever so slightly. The ganzfeld is set up so that an interpretation must be made of a verbal report from the test subject to be matched against an image allegedly sent telepathically to the subject. Thus, even if an image bears little resemblance to the verbal description, if it is selected as the one most closely resembling the target then it counts as a hit."
Mr. Carroll, this is why there is a 25% chance hit rate! If the descriptions the test subjects gave did not match the targets, then the law of chance says that a purely random hit would be expected 25% of the time, not 33.2%! The odds against chance of randomly getting a hit rate 8.2% above the 25% chance hit rate for 2,549 ganzfeld sessions are, as you correctly quoted Dr. Berger, "a million billion to one"! Please reflect on that (if only for a moment) Mr. Carroll.
Jason Ewing
reply: I did. I gave your comments the moment they deserve. I'm probably wasting my time, but let's try this.
Imagine that these pictures are the targets. Let's call the one trying to receive telepathic messages the receiver and the one who sends the messages the sender. The proper way to do this study would be to tell the receiver that the sender will be concentrating on one of these four pictures for a specified amount of time (agreed upon by all participants as sufficient to accomplish this mission). A specified number of trials should be established before the test begins. The targets will be randomly assigned. At the end of the specified time for each trial, the receiver will be signaled electronically to write down his response. Over a long series of trials, a success rate of 25% would expected by chance.
However, in the ganzfeld experiments the receiver reported orally what impressions he was getting. If he said Washington, D.C., that would count as a hit (close enough to Abraham Lincoln). Verbal reports should not be used in these tests because there is no way to specify before the trials begin what will and what will not count as a hit.
Some tests have the receiver draw a picture based on the "impressions" he or she is allegedly receiving. Then an evaluator or team of evaluators looks at the drawing and decides whether it's close enough to call a hit. Again, this method is flawed because there is no way to specify before the trials begin what will and what will not count as a hit.
However, if my advice were followed, then it wouldn't be a ganzfeld test but a simple telepathy or remote viewing test.
-------
Jason replies:
I must begin my reply to your comments by once again expressing my bafflement, and by noting that you seem to have an aversion against reading the materials you reference. Since, if you had read even Ray Hyman's skeptical assessments, you would know just how ridiculous your so-called criticism is. You begin your reply to me by giving four examples of targets and then stating: "The proper way to do this study would be to tell the receiver that the sender will be concentrating on one of these four pictures for a specified amount of time (agreed upon by all participants as sufficient to accomplish the mission)." That statement illustrates your profound ignorance of the Ganzfeld. A person who has only the most basic knowledge of the experiment knows that the Ganzfeld is not forced choice (as you seem to think), but free response. The difference being that in a free response experiment neither the "receiver" nor the "experimenter" know what ANY of the pictures are. In the Ganzfeld, the "sender" (who is locked in a sound proof room) has a target randomly selected at the beginning of the test and attempts to transmit it via telepathy to the receiver. It is not until after the session that four pictures, one of which is the correct one, are presented (by the still blind experimenter) to either the receiver or an independent judge. Both of whom are also completely blind to the target. Your ignorance of these basic facts is telling. You then go on to add that if a target was Abraham Lincoln and the receiver reported he or she saw Washington D.C., then it would likely count as a hit and that: "Verbal reports should not be used in these tests because there is no way to specify before the trials begin what will and what will not count as a hit." Mr. Carroll, what will count as a hit is if a completely target blind receiver believes that he or she has been getting an impression relevant to the correct target and chooses the correct target. Or if an independent judge who (as I will not fail to emphasize yet again) is completely blind to the target is compelled, based on the transcribed verbal account given by the receiver, to choose the correct target. Even taking into account your absurd ignorance of the correct methodology, it is still quite hard to see why you think it matters whether a receiver says he is seeing Abraham Lincoln or Washington D.C. I suspect that you think that since (according to your complete ignorance of the methodology) the receiver knows all four targets before hand he or she would be likely to choose one before the session starts and imagine relevant images connected with that picture. Yet even if the receiver knew what the four pictures were before hand this still wouldn't account for him or her getting above chance hitting. The reason the receivers in the Ganzfeld pick the correct target more than chance would allow is because images, which are relevant to the target, often appear in his or her mind during sessions, compelling the receiver to choose correctly 33.2% of the time. How is that not evidence for telepathy? So in closing, instead of trying to explain the results of the Ganzfeld by showing how fraud, sensory leakage, or poor randomization might account for the results, you (perhaps aware that those explanations fall short) come up with a nonsensical criticism which does nothing to explain why the Ganzfeld hit rate deviates so significantly from chance. Mr. Carroll, I sincerely hope you go and read the material that you referenced before replying to me again. You do your fellow skeptics a great disservice by being so uninformed.
Jason Ewing
reply: Jason is right about my suggestion not being relevant to doing a ganzfeld experiment. My point is simply that any hit rate of 33.2% rather than 25% would be much more difficult to explain by reference to such things as sensory leakage, common themes, poor randomization, or other methodological problems if another type of test is done. The ganzfeld has its reasons, I suppose, and it is probably pointless to suggest to the true believer that there might be something besides telepathy being measured by these experiments. Jason says that The reason the receivers in the Ganzfeld pick the correct target more than chance would allow is because images, which are relevant to the target, often appear in his or her mind during sessions, compelling the receiver to choose correctly 33.2% of the time. Perhaps. But this is not the only explanation.
ganzfeld experiments
