A Collection of Strange Beliefs, Amusing Deceptions, and Dangerous Delusions

From Abracadabra to Zombies

reader comments: Alternative Health Practices

31 Oct 1997
Dr. Carroll:

My name is Craig Nelson. I'm a chiropractor and a clinical researcher and I teach a class in critical thinking to chiropractic students. Really. You may have stumbled across my name in some of Drs. Barrett's and Jarvis's writings on alternative health care and chiropractic. My criticisms of chiropractic are often quoted by those in the skeptic community to support their points of view. This background info is intended to demonstrate that I am a dues-paying skeptic. (I'll heed your admonition about unsolicited manuscripts, but if you'd like to read some of my essays I'd be happy to send them to you.)

Your site is superb. It has been a regular stop of mine since I found it. Your entries on alternative health care and chiropractic are informative, accurate, and balanced. I regularly recommend your site to my students.

However (naturally, there's a "however"), I think it is incomplete and incomplete in a way that betrays a common misconception about the nature of science, health care, and medicine. This misconception seems to be shared by most skeptical observers of health care. One of your correspondents asks why you don't have entries on chemotherapy and radical mastectomy in your dictionary and why you only discuss alternative health care. Your reply is that, "I discuss only alternative medicine because of its general pseudoscientific or unscientific approach. Alternative medicine is generally based upon untested claims and is mainly appealing to desperate people." While you do discuss the imperfections of medicine your reply implies that medicine does not suffer from these shortcomings. This requires some examination.

One of the characteristics of most alternative health care systems is that they are predicated on some loony metaphysical or pseudoscientific principles, as you suggest. And medicine is not predicated on such principles, not an unimportant distinction. However, this distinction does not immunize medicine from the possibility of being fundamentally flawed.

Over the last decade or so there has been something of a revolution in the medical/scientific community that has concluded that medicine is deeply flawed. (I hasten to note that what I am describing originates within medicine, to its credit, and is not the product of the musings of Deepak Chopra and others.) This revolution is described, variously as "Outcomes-Based Medicine," or "Evidence-Based Medicine." The first article in the bibliography from JAMA is a good primer in the principles of this movement. That article comes to the following conclusions:

1. Medical education and the practice of medicine have been, and are currently, based primarily on tradition, common sense, uncontrolled observations, and the wisdom of elders. These are all unreliable predictors of patient outcomes.

reply: The tradition upon which medical education and practice are rooted is the study of anatomy, physiology, biochemistry, etc. I have no idea what you mean by saying that studying these sciences is an "unreliable predictor of patient outcomes." Also, Common sense is too vague an expression to comment on. As for the "wisdom of elders" being relevant to patient outcomes: it depends on the elders. Most physicians do some sort of apprentice work at a hospital before they are set loose upon the general public. Their teachers at medical school might be considered as elders passing on the wisdom of their disciplines. If the study you cite actually claims that having done an internship and gone to medical school are "unreliable predictors of patient outcomes," I would ask myself what does that imply? That people with no medical education or training have equal or better patient outcomes than those who do? That education and training are irrelevant to patient outcomes? Or that some number cruncher has figured out a way to come up with a formula which can find no significant correlation between scientific education and training and "patient outcomes"?

But the item I want to address most is the one which refers to "uncontrolled observations." I have heard this criticism from other chiropractors and it has generally been based upon a profound lack of understanding about the nature of control studies and post hoc reasoning. This may not be true in your case, but I believe the misunderstanding is widespread among your colleagues and is worth commenting on.

Let me start with a non-medical example. If I have a bag of coffee beans before me and a coffee bean grinder and I want a cup of coffee, I will grind the beans. I have done this for years. I know it works. I do not have to put whole beans into the filter and pour water over them to know that I will not get the cup of coffee I desire. I do not have to do a control experiment to prove that grinding the beans is a necessary condition for making a proper cup of coffee. I don't need to make two pots of coffee, one with ground beans and one with unground beans, nor do I need to do this many times to prove I can repeat the experiment with the same results. Control experiments are only needed when there is good reason to doubt the causal relationship of two factors. It is not necessary to do a control experiment to establish reasonable belief about a causal relationship in those cases where background experience and knowledge provide sufficient evidence for the belief. The experience and knowledge can, and often are, analogical. That is, even the first time I wanted to make a cup of coffee from beans, I could have reasoned by analogy that I would have to grind the beans in order to make the coffee. I also could have been taught by another person who showed me how to make coffee by grinding the beans. Again, I would have had enough knowledge and experience to recognize that grinding the beans is necessary; there would have been no reasonable requirement to do a control experiment before making coffee on my own.

Now, let's switch to a medical example. Let's take a medical procedure understood by almost everyone: the setting of broken bones. Some chiropractors would say that this is an example where medical doctors have not done controlled experiments, so how dare they criticize "alternative" practitioners for handing out remedies and doing therapies that are untested! Medical doctors do the same thing! Non-sense. There is no significant comparison between giving someone a back manipulation to cure an ear infection and setting a broken bone. The fact that neither procedure has gone through clinical trials is a trivial fact in the argument that medical doctors use "uncontrolled observations." The knowledge and experience which leads physicians to set broken bones is vast and supportive; the knowledge and experience which should lead anyone to think that spinal manipulation would cure an ear infection is slim and counter-intuitive. Some procedures need testing; others do not. One does not need a lot of wisdom to tell the difference, but one does need to have some skill at critical thinking.

2. There is a belief that a deep understanding of basic physiologic and pathophysiologic principles will lead invariably to effective interventions even in the absence of direct evidence of patient benefit. This also turns out to be a very unreliable predictor of patient benefits.

reply: I am not sure what the ambiguous "this" refers to, but let's assume it refers not to the belief but to the fact of understanding principles of physiology. I take it that your claim is that knowledge of physiology does not correlate significantly with patient benefit. What follows from that? Quit studying physiology and the practice of medicine will be unaffected? If reading books on herbs correlates more significantly with patient benefit, should we then have physicians read books on herbs? Or does this exemplify Mark Twain's citation of Disraeli that "there are lies, damned lies and statistics!"

Other principles of evidence-based medicine can be summarized as follows:

1. The clinical effectiveness, (i.e., the extent to which patients actually benefit) of most (>50%) medical procedures, both diagnostic and therapeutic, is unknown. (Some estimates of the percentage of medical procedures with known effects are as low as 15%.)

2. When the effectiveness of common medical procedures is actually measured in clinical trials the results very often show those procedures to be useless or harmful. (The list of commonly used medical procedures that have been shown in the last decade to be useless or harmful is very long. Many of these procedures are still in use.)

reply: This claim is false. Where do you think those statistics about your chances of recovery from, say, breast cancer through surgery and chemotherapy come from? If you mean, that no one has done a study to show that it is better to set a broken arm than to let it be, then the statistic you quote is of little importance. I've heard similar complaints from chelation therapists who compare chelation therapy for blood circulation and heart problems with bypass surgery and angioplasty. Where are the control studies of people who are given bypass surgery or angioplasty compared with a control group? To ask the question indicates a profound ignorance or conscious attempt to deceive the public. To pretend to do bypass surgery or angioplasty so that you can compare the poor dupes given the fake medical procedure with those given the real procedure would not serve medical science or knowledge. Comparing patients who have surgery with those who do not is not completely unreasonable, but it is not completely reliable, either, since individual patients will have their individual differences. Nevertheless, when a surgeon tells a family member that if a particular operation is not done, the likely outcome is "a", "b" or "c", that surgeon is not just guessing (unless he or she is a fool, and medicine has its fools, as every profession has). This does not mean that surgeons are infallible, nor does it mean that they have not, from time to time in their history, performed a number of unnecessary surgeries. Nor does it mean that even when a practice is based on sound theory, that the practice is necessary or even good. The answer to these errors is to correct them. It does not follow from the fact that surgeons have in the past advocated unnecessary surgery that all surgery should be stopped, or that we should not allow those surgeries which are based only on sound theory, experiments on animals and apparent clinical successes with many satisfied customers.

On the other hand, chemotherapies should be, and are, clinically tested, not on humans (except in desperate circumstances) but on other animals. Chelation is more like chemotherapy than surgery. Chelation seems to be based on unsound theory, but it is testable and should be tested, and has been tested, using control studies. When those studies pile up a preponderance of the evidence that the therapy is ineffective it should be dropped from the list of treatment procedures. Why do chelation therapists continue to sing the praises of a treatment which has exceedingly more evidence against it than for it?

3. Patients, and more importantly, practicing physicians, are generally unaware of principles 1&2;.

reply: I have no doubts that most patients are unaware of these controversies, but I would like to see the evidence that most physicians are unaware of studies done in their areas of practice which show that common procedures are ineffective or harmful. This assumes that the vast majority of surgeons, for example, know nothing about criticisms of their practices. This may be, but I'd like to see the evidence for such a claim.

4. Medical training does not prepare practicing physicians to recognize sources of error in clinical decision making and does not prepare them to critically evaluate the scientific literature. (Physicians are not scientists. Indeed, they often tend toward the naive and credulous end of the spectrum. They tend to accept, uncritically, what is told to them or what they read.)

reply: hmmm. And your source for these claims is an article in the Journal of the American Medical Association. I would guess that such a claim in one of the profession's most prestigious journals would have been met with an uproar of disapproval by the association's members. Please send me the literature on the study which demonstrated that physicians "often tend toward the naive and credulous end of the spectrum. They tend to accept, uncritically, what is told to them or what they read."

5. The clinical studies upon which many treatment standards are based are often of such poor quality that those standards have very little validity.

I don't expect you to take these assertions at face value, but I honestly don't think they would be disputed by those in medicine who have studied these issues. The attached bibliography should give you a good sense of the problem, if you are interested. Again, please note that all these sources are from the most respected medical journals.

reply: Perhaps one or two physicians might read your comments and respond. You are incorrect in your expectations, however: I do take your assertions "at face value." On their face, they are preposterous and incredible.

So, in place of the chiropractic's Innate Intelligence, acupuncture's meridians, and homeopathy's magic water, medicine has been based on an unscientific principle which, although un-stated, is in force. That principle could be expressed as:

"If a procedure has been used for a long time, and if that procedure seems to make sense, and if that procedure is favored by old doctors who teach young doctors, and if that procedure does not directly violate known biological principles, then it can be assumed that that procedure benefits patients even when there is no direct evidence of such benefit."

You have characterized alternative care as untested and unscientific. Those adjectives are also accurate with regard to medicine.

reply: and you, sir, have created a straw man with your characterization of medical science and practice. Do you really think any profession could so dupe the entire world that for years it could use procedures on people "when there is no direct evidence of [patient] benefit"? I wouldn't even make such a claim about your profession or even about religion. I think, however, I have examined your claims above and have no more to say of them except to repeat that they seem to be based on a profound lack of understanding of the purpose and nature of control studies.

Back to the original question--why not subject medicine to the same type of scrutiny as alternative care? If your answer is that medicine is not afflicted with any systematic or pervasive shortcomings that threaten the health or pocketbooks of patients, and that such scrutiny is wasted, you are mistaken. Indeed, I think if one framed the question as follows--Under what circumstance and with regard to what sort of health care will a skeptical and critical perspective yield the greatest advantages in terms of preventing harm or economic loss?--the answer is under the circumstance of being a patient in the mainstream health care system.

reply: medicine suffers from the same shortcomings of all human forms of knowledge; it is fallible. It also is correctable. This cannot be said of those systems of thought which are fundamentally metaphysical in nature. In those systems of thought, the basic claims are not testable and can therefore never be proven incorrect. Hence, once they get established they tend to become dogmatically adhered to and never change. The only way to change dogma is to become a heretic and set up your own counter-dogma.

Much was made of Eiseneberg's finding in the New England Journal that $13.7 billion is spent annually on alternative health care--a lot of money to spend and waste on New Age bunk. But if that is a lot of money to waste, what of the over $1 trillion that is spent annually on conventional health care? If the above principles are even partially true it suggests that a very large percentage of that trillion is wasted. How much? Who knows, but a sum that will certainly dwarf the 13.7 billion spent on alternative care.

reply: If your claims are even partially true (whatever that means), it may suggest to you that "a very large percentage" is wasted, but to me it does not suggest any such thing. I am willing to grant that a great deal of money is wasted on medical procedures, including unnecessary diagnostic tests, duplicate and fraudulent billings by immoral physicians, and extraordinary measures taken to preserve the bodies of brain dead humans, etc. I have no idea what percentage of the total bill this amount to. Are there immoral and unethical medical doctors? Yes. What does this prove? Not that immorality is endemic to the medical profession, but that medical science is a human science, and anything humans are capable of, so are physicians. This is not a scoop.

A final thought. I think that the reason that skeptics have failed (and I think it is a failure) to properly address the issue of skepticism and health care is that they have confused the problem of scientific illiteracy and the problem of ineffective and unsafe health care. The appeal of mystical and nonsensical approaches to health, as is typical of alternative care, is certainly troubling evidence of scientific illiteracy, and it is a legitimate undertaking to oppose those practices. But while these alternative practices are more obviously and overtly unscientific, it turns out that the more subtle scientific shortcomings of medicine have a far greater impact on the public's health and well being.

reply: Obviously, you have not persuaded me of these "subtle scientific shortcomings."

In your reply to the gentleman who asked the question about chemotherapy and mastectomies, you stated that wearing a pyramid hat and chemotherapy are not in the same class. Indeed, they are not. At least in some cases of chemotherapy or other types of conventional cancer treatment, the pyramid hat is to be preferred. If I am someday afflicted with prostate cancer and I treat that cancer with a pyramid hat, I will look stupid and my cancer will be unaffected. If, on the other hand, I treat that cancer conventionally with surgery, radiation, and chemotherapy, it is highly probable that the cancer will be unaffected, and I will be impotent and incontinent. But I won't look stupid.

reply: that is a matter of opinion.

There is obviously much more to be said on this topic, and if you are interested I would very much like to hear your thoughts.
Craig F. Nelson, D.C.

BIBLIOGRAPHY (provided by Craig F. Nelson, D.C.)

Evidence-Based Medicine Working Group. 
Evidence-Based Medicine: A New Approach
to Teaching and the Practice of Medicine.

McCormick J.  Health Promotion: the ethical
dimension. The Lancet.1994;390-391.

Fitzgerald F.  The tyranny of health.  NEJM.
1994; 196-198.

Becker M.  The tyranny of health promotion.
1986; Public Health Reviews.15-23.

Relman A.  Assessment and Accountability: The
Third Revolution in Medical Care. NEng J Med:

Smith R.  Where is the Wisdom? The Poverty of 
Medical Evidence. BMJ. 1991:798-799.

Schoenbaum S.  Toward Fewer Procedures
and Better Outcomes. JAMA.1993:794-796.

14 Nov 1997: Craig F. Nelson, D.C. replies

I'm torn between responding in the polite and respectful tone of my original letter or in a tone more in keeping with your smug, sarcastic, and self-satisfied reply. We'll see. I may oscillate back and forth.

You make several references to my lack of understanding of "control studies." (The term is actually "controlled studies," but never mind). Sir, I am a clinical scientist, you are not. Your profound ignorance of this science is evident in every paragraph of your reply. Or am I wrong? Perhaps you do know the meaning and relevance of terms like confidence interval, p-value, correlation coefficient, cohort study, prospective clinical series, Cronbach's alpha, type I error, type II error, Bonferroni correction, and cross sectional study. If you are not familiar with these concepts you should correct that deficiency or refrain from offering your uninformed opinions on the scientific merits or shortcomings of medicine. You are not competent to do so.

Reply: Apparently, there has been a grave misunderstanding. I didn't know we were debating the merits of particular clinical studies. I thought we were discussing a philosophical issue: the nature of science and its relationship to "alternative" and "standard" medicine.

You completely misinterpreted the overall theme of my letter, which theme is blindingly clear to anyone who can for an instant transcend the inertia of their own biases. I did not at any point defend any alternative health care practice (except facetiously in the case of a pyramid hat for prostate cancer) on basis that because medicine is flawed it's OK for alternative practices to be untested and unscientific. My point was that most alternative practices are bunk and that medicine is also deeply flawed. Get it straight.

Reply: I'm glad we got that straight. Or have we? When you say "medicine is also deeply flawed" I take you to mean much more than that medicine is fallible, uses questionable procedures at times, that some doctors experiment on their patients using untested or discredited procedures, that many medical procedures are approved by some physicians but discredited or rejected by others, etc. I take you to mean that "standard" medicine shares some kind of fundamental flaw with "alternative" medicine, such as the requirement for the patient to share the practitioner's metaphysical beliefs for the cure to work; or the idea that a single type of treatment is a panacea and can cure many, completely unrelated types of disorders; or that insight and intuition are preferable to scientific testing of therapies and procedures. I didn't realize that you and I are in complete agreement about one aspect of scientific medicine: it is a fallible, human activity, and as such makes many errors. However, I see scientific medicine as essentially able to correct its errors. I don't attack and criticize "alternative" therapies because they are fallible or incorrect, but because they are essentially incapable of correcting errors. This is often due to the fact that such therapies are based upon metaphysical assumptions which cannot be tested. It is also often due to the dogmatic refusal of proponents of "alternative" therapies to give up their ideas in the face of overwhelming evidence to the contrary of what they believe.

In traditional medicine, I do not see perfection, but I see errors being admitted and attempts to correct them as part of the fundamental nature of the enterprise. I don't see the attempt to improve and weed out error, or the frequent admission of error, as being fundamental to "alternative" therapies. As just stated, such therapies tend to be pseudoscientific on one of two counts: either they are profoundly metaphysical or they dogmatically refuse to accept empirical evidence which falsifies their claims.

The untested/discredited procedures to which I refer are not the setting of broken bones, but the use of medication to treat hypertension, high cholesterol, or migraine headaches; the use of MRI to diagnose back pain and headaches; the use of fetal monitoring during labor; the use of antibiotics to treat otitis media, the use an annual physical exam to screen for disease, the use of anti-arrhythmia drugs following heart attacks, the use of various cancer screening tests, and of countless other procedures. If all this is news to you this is a function of your own ignorance and not of my misunderstanding. (Just curious. Have you ever actually read a clinical study?) And don't give me the line that all these things are approved by the FDA and must therefore by valid. It is infinitely more complex than that, but to understand the complexities you would again have to know something about clinical science.

Reply: Each of the procedures you list, and the way in which scientific medicine deals with them, will illustrate my point: there will be disagreement and controversy, error and argument, testing and more testing, etc. Decisions will be made by fallible human beings engaging in the fallible practice of scientific medicine. Some of those decisions will be bad decisions, but in time they will be discovered for what they are and treatments which were once standard will be rejected and replaced with other treatments. Medicine will grow, it will progress, it will change dramatically. Homeopathy, iridology, reflexology, aromatherapy, therapeutic touch, etc., will not change in any fundamental ways over the years. This is not a scientific issue, requiring mastery of scientific jargon to comprehend. This is a philosophical issue.

You seem to be suggesting that I should be criticizing these "untested/discredited" procedures, yet you also make it clear that you think I would need your kind of training in clinical science to be able to intelligently evaluate these procedures or any studies regarding them. So, it seems there is no way I could please you. I am not fair because I do not criticize the studies and procedures of scientific medicine but I am not competent to evaluate them, either.

I particularly enjoyed your objection that the JAMA article could not really say what I said it did because AMA members would have really been outraged. Now there's a logical gem! I won't make any further effort to explain these issues to you. If want to understand them you must actually make an effort and read these papers. You'll find them in the library--that's the big building with all the books and journals. It does not take great intellectual courage or skill to take a stand against healing crystals or psychic channelers. I'll bet you've come out firmly against the Easter Bunny as well. But if you want to engage a problem that is more interesting, important, nuanced, and requires more subtle and sophisticated reasoning skills to understand, you might consider the issues I have raised.

Reply: Actually, what I wrote was "I would guess that such a claim in one of the profession's most prestigious journals would have been met with an uproar of disapproval by the association's members." Using the straw man seems to be a favorite tactic of yours. You need to work on your ad hominem attacks, though. It is not to your advantage to make your opponent seem too pathetic. What glory can there be in winning a battle of wits with an opponent you declare to be unarmed?

larrow.gif (1051 bytes) Alternative Health Practices

All Reader Comments


This page was designed by Cristian Popa.