N'kisi & the N'kisi ProjectRupert Sheldrake replies to the N'kisi entry and I respond Robert Todd Carroll - June 13, 2007 |
Robert
replies to Rupert Sheldrake
(Ian Stevenson was one of the founders of the Society for Scientific Exploration and its Journal of Scientific Exploration. The latter, Stevenson wrote, was to "provide a forum where research on paranormal phenomena can be presented to other scientists without obstruction or derision."*) |
Rupert replies to Robert Todd Carroll* On his Skeptic’s
Dictionary site, http://skepdic.com/nkisi.html Robert Todd Carroll
comments on some experiments that Aimee Morgana and I carried out with her
parrot N’Kisi, a bird with an astonishing vocabulary (currently over 1,200
words) and remarkable linguistic abilities, often making meaningful comments
and speaking in sentences. |
ad hominem
ad hominem Personal attacks but no substantive criticism here. |
Testing a Language - Using a Parrot for Telepathy Carroll is a committed skeptic who is strongly motivated to try and discredit the positively and statistically significant results of these tests, which imply some form of unexplained communication between Aimee and N’kisi. Carroll devotes much of his discussion to the details of the statistics and the experimental methods, but then concludes dismissively, “My devout wish is that when such studies as these are published in the future, responsible journalists continue to ignore them and recognise them for the rubbish they are.” This is not the wish of a skeptic interested in open-minded inquiry and free discussion, but of a committed ideologist who wants to censor what the public gets to know. |
straw man:
these comments were not about his paper but about a website featuring the
parrot. My criticisms of the paper are not based on the claim that the
independent judges could not understand the parrot. poisoning the well
Still no substantive criticism.
I don't say there wasn't. |
In his criticisms of research with N’Kisi, Carroll tries to persuade his
readers that the speaking of this bird is so indistinct that it can only be
interpreted after people have been told what to expect. He says that the
bird sounds are “gibberish until you are told what to look for”. This is
untrue. If he had read our paper he would see that our method did not rely
on people knowing what to look for. The tapes of our experiments with N’Kisi
were transcribed independently by three different people, who worked blind.
They did not know what targets were involved in the experiments, or when one
trial ended and another began. Nor did they know what words were on the
pre-specified list of key words that could count as hits or misses. There was a remarkably good agreement between the transcripts of the three independent transcribers, and they also agreed well with Aimee Morgana’s own transcript (which was not included in the analysis because it was not blind). For an example of the results of these for a comparison of these four transcripts, see our Table 1. [link] The three blind transcribers agreed completely on 105 occasions on which target words were said. Two out of three transcribers agreed on a further 12 occasions (see our Table 4.I). |
No, you didn't
analyze the issue of the times when the parrot was silent or said things not
on the list, sixty trials or 40% of the trials (see below for more on this
topic). Nor did you analyze the odd fact that almost one-third of the images
and more than half the hits came from just 2 of the 19 pictures. You do not need to be a trained scientist to recognize that when testing the telepathic ability of a parrot and it says nothing when the subject looks at a picture card that it should be taken as a sign that no telepathy is occurring. Do you really expect the public to believe that you could add sixty more instances of failure to your tests and still arrive at the same (low) level of statistical significance? I recognize that the data will favor your hypothesis if you assume that during the silent periods the parrot is to be excused because of his short attention span. |
In order to analyse the data as objectively as possible, we analysed all aspects of the results taking into account the data where all three transcribers were in agreement, or when only two agreed on a word, or when only one reported a particular word. In all cases the results were highly significant statistically (see our Table 4). [link] |
repeats the
straw man noted above: I make comments about this tape, but they are
distinct from my criticisms of your research paper. (Note: the article which
contains the criticism of Sheldrake's paper is not devoted to that single
issue. It is broader in scope. For Sheldrake to use my comments about one
thing (the tape) to criticize my comments about something else (his research
paper) is irrelevant. poisoning the well Again, nothing of substance here. |
Carroll bases his comments on a short extract from a tape recording of N’kisi posted on my web site. His contention that people can only understand what N’Kisi is saying after they know what to listen for goes against the evidence of our published study. Of course he is free to dismiss with contempt any research he doesn’t like, but he is wrong to mislead his readers by pretending to be scientific. |
Maybe so, but
it is a false analogy to claim that your test of a telepathic parrot
requires the same methodology as testing young children, the autistic, and
the otherwise impaired. What may be justified in these other situations does
not appear to be clearly justified in this particular type of test. In fact,
it seems clearly inappropriate. Your assumption that the same methodology
should be used here is questionable. Why should we accept Sheldrake's
assumption that the parrot turns his telepathic interest off and on, and it
is on only when the parrot uttered a word on the key list? The parrot is being tested for telepathy. If you are going to accept his "answers" when he talks but not when he is silent, you should have a better justification than that is how you would do it if you were testing autistic children, toddlers, etc.. |
In analysing our results, we excluded trials in which N’Kisi did not say anything, or did not say one of the pre-specified key words on which the analysis was based. This is a standard procedure in experiments with young children and animals, who cannot be forced to respond, or may not respond in some trials at all, owing to their shorter attention spans and inability to understand the testing situation. There are many examples of this in the developmental psychology literature, for example in testing autistic children, toddlers, and others who cannot be expected to respond to tests exactly like adult humans. Carroll seems ignorant of research methodologies in these fields. He implied that that our method was scientifically improper: “Some might argue… that by ignoring so much data where the parrot clearly did not indicate any sign of telepathy is strong evidence that Sheldrake was more interested in confirming his biases than getting at the truth.” |
ad hominem In fact, I note in my article: "The other reviewer accepted Sheldrake's observation that even if you throw out the flower data, you still get some sort of statistical significance. This may be true. However, since the bird allegedly had a vocabulary of some 950 words at the time of the test, omitting sessions where the bird said nothing or said something not on the key list, is unjustifiable."
|
Carroll’s comment is deliberately misleading. As Carroll must know if he read our paper, an independent statistical analysis has shown that counting all trials and including all data does not significantly alter the results. When our paper was reviewed, one of the referees thought that we should not have omitted instances in which the parrot said nothing or did not utter one of the pre-specified key words. His comments were published in the Journal of Scientific Exploration immediately after our paper. Having expressed this concern, he continued, “I therefore requested data on the omitted cards/phrases, which the authors immediately supplied. I did a permutation test on the entire data set and, and found p-value that differed only trivially from the one stated in the article. Although the authors have done an analysis that I would not have done (by omitting data), it makes no difference to the results.” |
My point in
making this suggestion was that there is no evidence that it is reasonable
to assume that when the parrot is by itself uttering words that it is trying
to communicate telepathically with Morgana. Sheldrake's study is based on
this questionable assumption.
We agree on one thing, at least.
|
Carroll also suggested that we should have done a baseline study in which
the parrot was videotaped for two-minute periods outside the context of the
experiment, and suggested we should have made hundreds of such clips and
then compared them to the clips we used for the analysis. This methodology
would certainly have been objected to by more scientifically-minded skeptics
than Carroll, who would have questioned the basis on which we decided to
make these control clips. For example
if they had been at random times, many
of them would have been when the parrot was asleep and not saying anything.
If we had made them during the day when Aimee Morgana was “doing something
unrelated to the key-word pictures,” then we would have been accused of
selecting activities that would have biased the results. What Carroll fails to appreciate is that we used a randomised permutation analysis to assign the two-minute clips at random to the photo-card stimuli, in order to find out how many hits would have arisen by chance. This kind of statistical analysis is also known as a Monte Carlo simulation, because it involves numerous randomized permutations to replicate the element of chance, using computers. In this analysis, following standard, well-established methods, all utterances of key-words by the parrot during each two-minute period were randomly assigned to each stimulus photo-card to see what were the chances of the parrot getting a hit by coincidence. The analysis involved 20,000 random permutations, and showed that N’kisi said words corresponding to pictures that Aimee was looking at very significantly more than could be explained by chance (p= 0.0003; our Table 4). Carroll repeats his criticisms in an even more extreme and biased form in an extraordinary attack on Jane Goodall, widely respected for her pioneering work on chimpanzee behaviour: http://skepdic.com/refuge/bunk33.html “Goodall's reputation will not be enhanced by her belief in Bigfoot or animal telepathy. If she considers Sheldrake a good scientist, then one must wonder what standards she has held herself to over the years in her studies of chimpanzees.” |
Nobody is above criticism, not Rupert Sheldrake and not Jane Goodall. Even great scientists are prone to confirmation bias. Sheldrake's comments about Goodall's reputation and accomplishments are an irrelevant appeal to authority. Note: I have a lot more to say about Goodall than just the clip cited by Sheldrake. Does Sheldrake really think that people who win prestigious awards get a free pass to say whatever they want without fear of rebuttal? |
Jane Goodall’s 40 years of research with chimpanzees has been presented in
many books, scientific journals and films. Her groundbreaking observations
of chimps using tools have been filmed and replicated by independent
observers. She was awarded the Legion of Honor, is a Dame of the British
Empire and has received many other prestigious awards honoring her
scientific discoveries. |
Sheldrake
is consistent. He began with an ad hominem and he concludes with another
personal attack. He keeps attacking my motives rather than my arguments. If
I had the credentials Sheldrake has and designed such poor experiments and
couldn't do any better than a personal attack on a
critic, I'd hang my head in shame. Sheldrake is proof, in any case, that even people with
scientific credentials get carried away by their strong beliefs and dogmatic
zeal. What does Sheldrake have to say about one of the peer reviewers who thought that the fact that the flower word and picture played so heavy a role in the outcome that the paper's results were distorted and that the paper should not be published? What does Sheldrake have to say to the editor who published his paper with reservations? The editor noted: "My personal reaction is that, once again, we have suggestive results, a level of statistical significance that is less than compelling, and the devout wish that further work with refined protocols will ensue." |
Carroll has no scientific credentials, and he gets carried away by his strong beliefs and dogmatic zeal. His style of analysis is amateur and pretentious; his intentions are polemical. If The Skeptic's Dictionary site were subject to independent peer-review, like articles in scientific journals, much of it would not survive. |