Ben Goldacre’s March 1st Bad Science piece for the Guardian, Don’t laugh, sugar pills are the future, in which he comments on the latest research to show that SSRIs are not much more effective than placebo in treating depression is, as usual, a bit thin on the ground with the actual science itself, even if his title might turn out to be remarkably prescient. And, wonder of wonders, I even agree wholeheartedly with a substantial amount of his earlier piece on February 27th, based on the same study, A quick fix would stop drug firms bending the truth. But far more interesting is the piece by Clive Cookson in the FT, Is there an ethical way to fine-tune the placebo effect?

In commenting on the recently-published meta-analysis of both published and unpublished trials of SSRIs led by Irving Kirsch at the University of Hull, which follows his more limited analysis of published studies in 2002, Cookson writes

Anti-depressants do not work, chorused newspaper headlines this week. The truth is quite different. The research in question showed that new-generation drugs, such as Prozac and Seroxat, relieve depression very well - but so do dummy pills.

This is the crucial point. Not, as Goldacre opines

It was fun to hear universal jubilation over the new meta-analysis showing once again that some antidepressants aren’t much cop in mild or moderate depression.

Cookson goes on to say

The study, led by Irving Kirsch at the University of Hull, is the latest testament to the power of the placebo. Analysing the drug companies’ own clinical trial data, the researchers found that four widely prescribed drugs improved patients’ score on the standard clinical test for depression by 9.6 points, while placebo pills gave a remarkable 7.8-point improvement.

As the researchers said in the journal PLoS Medicine: “The response to placebo in these trials was exceptionally large.” This set the bar for demonstrating efficacy so high that - except for severely depressed patients - the difference between treated and placebo groups did not reach a “statistically significant” level.

And continues

But researchers are only just beginning serious investigation of placebo power. “We do not really know what the mechanism is,” says Dr Derbyshire. “In fact, there may be lots of different mechanisms. For example, placebo painkillers somehow activate the brain’s endorphins [natural painkillers] while placebo aspirin activates a natural anti-inflammatory effect.“

Exactly. Now we’re getting a bit closer to the mark.

I’ve talked elsewhere about the non-local aspects of homeopathic treatment and the quantum mind-like effects observable and said

I’m saying the qualitative principles of quantum mechanics have the potential to model some of the observations which have been made in respect of homeopathy and many other of the more subtle, holistic therapies. Let’s get one thing straight right away. These effects are not specific to the therapy. They just become more apparent in the context of the therapy because of its level of subtlety and its holistic nature. They’ll be occurring just as much with conventional medicine too, but will be far less obvious to observers who are looking at things in a much more focused and linear way.

Here it would seem that evidence is starting to become clearer. And we need some new terms. ‘Placebo’ can’t be used to describe the specific effect of the patient’s expectation that the pill they’re taking will help them, at the same time as being a dustbin term for all non-specific effects of treatment. Personally I think it’s time the word placebo was restricted to its original sense and use: a dummy pill administered by a physician when he wants the patient to believe he’s taking the real McCoy. Using it in respect of intangible but verifiable effects of treatment is confusing the picture and leading to a derogatory attitude to these effects when we should be studying them free from such prejudice.

I’m going to suggest that there are 3 principle components in this effect: i) the effect due to the patient’s conscious expectations, ii) the effect due to the physician’s expectations transmitted to the patient through conscious entanglement, and iii) the effect due to conscious entanglement with the nature of the substance being prescribed.

The effect of patient expectation is clear and logical enough. Physician expectation also plays its part. In a February 2000 article in the Guardian on the rise of complementary medicine, Healing in Harmony, Jerome Burne wrote

Medicine is both an art and a science, but science has been firmly in the driving seat for 40 years. The arrival of CM practitioners may allow some of the more intangible aspects of the healing profession to re-emerge into the light, such as the power of the doctor’s own belief. “When I was starting out as a doctor, my professor told me about a new migraine drug,” recalls Marshall Marinker, professor of general practice at the University of London. “I prescribed it to a number of my patients, and it worked brilliantly. Many were completely cured. But then I began thinking about its mechanisms and how to design a trial, and it somehow stopped working so well. I never again got such good results as when I totally believed in it. I don’t think you can measure that sort of thing in clinical trials, but it is absolutely vital to the way medicine should work.”

The final effect, conscious entanglement with the nature of the substance being prescribed, is possibly going to be a harder stretch for some. Yet it seems the most plausible mechanism to explain how, as Derbyshire says above, “placebo painkillers somehow activate the brain’s endorphins while placebo aspirin activates a natural anti-inflammatory effect”.

And as it happens, this hypothesis also posits a rationale for homeopathy, explaining why effects should be observed when patient expectations aren’t relevant, and also why the wrong remedy has no effect.

So when Goldacre writes “Sugar pills are the future, if only there was a way to give them with integrity, and a straight face” he may very well find that the last laugh is, resoundingly, on him. There’s plenty of integrity and straight faces among homeopaths …

Evidence-based medicine is a worthy objective. According to the Oxford-based Centre for Evidence Based Medicine, “Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients. The practice of evidence-based medicine means integrating individual clinical expertise with the best available external clinical evidence from systematic research.”

The present campaign against homeopathy and other complementary therapies makes extensive use of these aims to justify calls for these therapies’ exclusion from the NHS on the grounds that there is “no evidence” for their efficacy. I’ve gone into the evidence question in relation to homeopathy in other posts. This one looks at the evidence base for biomedical treatment. After all, if biomedical treatment is to be so clearly preferred over the complementary therapies which presently lack the robust evidence base being demanded, then one would assume that biomedical treatments have a clear advantage here.

One would assume wrongly.

This from BMJ Clinical Evidence:

What proportion of commonly used treatments are supported by good evidence, what proportion should not be used or used only with caution, and how big are the gaps in our knowledge? Of around 2500 treatments covered 13% are rated as beneficial, 23% likely to be beneficial, 8% as trade off between benefits and harms, 6% unlikely to be beneficial, 4% likely to be ineffective or harmful, and 46%, the largest proportion, as unknown effectiveness (see figure 1). Dividing treatments into categories is never easy hence our reliance on our large team of experienced information specialists, editors, peer reviewers and expert authors. Categorisation always involves a degree of subjective judgement and is sometimes controversial. We do it because users tell us it is helpful, but judged by its own rules the categorisation is certainly of unknown effectiveness and may well have trade offs between benefits and harms. However, the figures above suggest that the research community has a large task ahead and that most decisions about treatments still rest on the individual judgements of clinicians and patients.

So let’s get this straight. Just 13% of the 2,500 treatments assessed — around 325 treatments — have been shown to have clear benefit. A further 23% are “likely to be beneficial”. The remainder, a staggering 64%, have either too many harmful side effects to be of clear benefit, are just plain ineffective and/or harmful, or have unknown effectiveness.

Yet so many of these drugs are presently being prescribed to the population of the United Kingdom that not only is the NHS close to bankruptcy, but traces of them are even showing up in rivers and drinking water supplies, including cytotoxic chemotherapy drugs which, according to figures publicised in 2003 by GlaxoSmithKline’s Allen Roses, have a response rate of only 25%. Further, for those that do respond to the therapy, a 2004 study concluded that “the overall contribution of curative and adjuvant cytotoxic chemotherapy to 5-year survival in adults was estimated to be 2.3% in Australia and 2.1% in the USA.” The authors went on to say “it is clear that cytotoxic chemotherapy only makes a minor contribution to cancer survival. To justify the continued funding and availability of drugs used in cytotoxic chemotherapy, a rigorous evaluation of the cost-effectiveness and impact on quality of life is urgently required.”

In the context of these figures, the present campaign against complementary therapies seems completely indefensible. Until biomedical treatments can provide robust evidence that their continued use is justified in terms of the benefit they provide to patients, then they too should be dropped from the NHS. And perhaps some of the enormous savings this would achieve could be spent on funding a better evidence base for complementary therapies …

I didn’t write this article. It’s from the website Suppressed Science. I’m posting it here in the interests of raising public awareness of this increasingly common and rather unpleasant condition. We’re already aware of its non-self-limiting nature — sufferers have clearly demonstrated their incapacity to limit its effects to themselves — and so far there’s no evidence of it being curable.

It should not be confused with scepticaemia, the condition of having doubt in the blood. Scepticaemia is essentially healthy. DD Scientismic fascistitis.

Contributions from other homeopaths on candidates for genus epidemicus remedies are welcome.

[The following article was inspired by an article by Dr. Robert L.Park, a spokesman for the American Physical Society, titled Seven Warning Signs of Bogus Science.]

Seven Warning Signs of Bogus Skepticism

The progress of science depends on a finely tuned balance between open-mindedness and skepticism. Be too open minded, and you’ll accept wrong claims. Be too skeptical, and you’ll reject genuine new discoveries. Proper skepticism must be careful not to throw the baby out with the bathwater.

Unfortunately, much of what comes out of the “skeptical” community these days is not proper skepticism, but all-out, fundamentalist disbelief. Such skepticism can be called pseudo-skepticism, pathological skepticism or bogus skepticism.

Here are seven major warning signs of bogus skepticism.

1. The Skeptic has reached her skeptical opinion not after careful research and examination of the claim, but simply based on media reports and other forms of second-hand knowledge.

Example: Pathological cold fusion skeptic Robert L. Park revealed in his March 1st 2002 What’s New column that Science was going to publish an article on Sonofusion, and that even though he had not seen the paper, talked to the researchers or conducted any personal research in the area, he already knew that the Sonofusion discovery would turn out to be “a repeat of the cold fusion fiasco”. Park used every bit of influence he had in a behind-the-scenes attempt to kill the paper. Luckily, the Science editor didn’t cave and decided to publish.

2. Making uncontrolled criticisms. A criticism is uncontrolled if the same criticism could equally be applied to accepted science.

For example, Park makes such a criticism in his book Voodoo Science (p.199). In the context of a discussion of an obviously pseudoscientific Good Morning America report on anomalous phenomena (debunkery by association: as if TV shows were the principal outlet for reporting the results of psi research!), Park writes

Why, you may wonder, all this business of random machines? Jahn has studied random number generators, water fountains in which the subject tries to urge drops to greater heights, all sorts of machines. But it is not clear that any of these machines are truly random. Indeed, it is generally believed that there are no truly random machines. It may be, therefore, that the lack of randomness only begins to show up after many trials. Besides, if the mind can influence inanimate objects, why not simply measure the static force the mind can exert? Modern ultramicrobalances can routinely measure a force of much less than a billionth of an ounce. Why not just use your psychokinetic powers to deflect a microbalance? It’s sensitive, simple, even quantitative, with no need for any dubious statistical analysis.

Where does Park’s assessment that effects that are only indirectly detected, by statistical analysis, are suspect, leave conventional science? Deprived of one of its most powerful tools of analysis. The cherished 1992 COBE discovery of minute fluctuations in the cosmic microwave background radiation would have to be thrown out, since it was entirely statistical in nature, and therefore by Park’s argument, ‘dubious’. The most celebrated discoveries of particle physics, such as the 1995 discovery of the top quark, or the results of neutrino detection experiments, or the synthesis of superheavy, extremely short-lived elements, would have to be thrown out, since they, too, are indirect and statistical in nature. Modern medicine would have to be invalidated as well because it relies on statistical analysis (of double- blind trials) to prove the efficacy of drugs.

For comparison: the American Institute of Physics’s Bulletin of Physics News, #216, March 3, 1995 gives the odds against chance for the top quark discovery as a million to one. A 1987 meta-analysis performed by Dean Radin and Roger Nelson of RNG (random number generator) experiments between 1959 and 1987 , on the other hand, shows the existence of an anomalous deviation from chance with odds against chance exceeding one trillion to one (see Radin, The Conscious Universe, p. 140).

Park’s argument is the quintessential uncontrolled criticism: accepted scientific methods that constitute the backbone of modern science suddenly become questionable when they are used on phenomena that don’t fit his ideological predilections.

3. The Pseudoskeptical Catch-22: “unconventional claims have to be proved before they can be investigated!” This way, of course, they will never be investigated or proved.

Parapsychology has been significantly hampered by this pseudoskeptical attitude. Pseudoskeptics complain that effect sizes are not bigger; but at the same time, they scream bloody murder if any grant-making agency even so much considers doing something about it. Radin writes in The Conscious Universe:

The tactics of the extreme skeptics have been more than merely annoying. The professional skeptic’s aggressive public labeling of parapsychology as a “pseudoscience”, implying fraud or incompetence on the part of the researchers, has been instrumental in preventing this research from taking place at all.

A similar situation exists in the new energy field. Pseudoskeptics like Robert L. Park are not content just dismissing things like cold fusion; they put massive pressure on policy makers and government to obstruct efforts to prove them wrong. Park’s successful lobbying of the US patent office to withdraw Randall Mill’s Black Light patent (which had already been approved) comes to mind as an example.

4. Evidence of refutal is anecdotal or otherwise scientifically worthless. Pseudoskeptics tend to accept conventional “explanations” for unconventional phenomena very easily, no matter how weak, contrived or far-fetched. A good historical example is the rejection of the crop circle phenomenon.

Doug Bower and David Chorley claimed in 1991 that they had created all of the British crop circles since 1978 (all 2000 of them). This was an extraordinary claim of the highest order. Two old men claimed that for over a decade, they have been creating circles and geometrical designs whose complexity defies easy geometrical construction in crops, but they were never able to demonstrate that they can do what they claim they could do. Any true skeptic would have rejected Bower’s and Chorley’s claim, since “extraordinary claims require extraordinary evidence”. Yet, the organized skeptics endorsed the claims enthusiastically and denounced the whole crop circle phenomenon a proven hoax.

5. The Skeptic rejects a discovery or invention merely because it has been believed for a long time that such a thing as the claimed discovery or invention is impossible.

This is the sole basis for the pseudoskeptical claim that, for example, a perpetuum mobile of the second kind is impossible. Park, for example, writes the following ignorant tirade in his 9/24/1999 What’s New Column:

PERPETUUM MOBILE: BETTING AGAINST THE LAWS OF THERMODYNAMICS.

Most free energy scams invoke outlandish new physics: cold fusion, hydrinos, zero-point energy, gravity shields, antimatter. But there are also throwbacks to the 19th Century that directly challenge the laws of thermodynamics. Physics Today carried a full-page ad for Entropy Systems, Inc. describing a heat engine that runs off ambient heat. It’s hardly a new idea. Two years ago Better World Technologies was touting the “Fisher engine” that violated the Second Law (WN 18 Jul 97). But it wasn’t new then either–it was the “zero motor,” invented by John Gamgee in 1880. It didn’t work then either, but Gamgee sold it to the U.S. Navy anyway.

Park’s sole argument appears to be that We Have Always Believed The Second Law Is Correct, So It Has To Be. Physicists who actually investigate this question without preconceived notions of what is possible or impossible have reached very different conclusions. D.P. Sheehan, A.R. Putnam and J.H. Wrighty of the University of San Diego write in a recent paper titled A Solid-State Maxwell Demon:

Over the last ten years, an unprecedented number of challenges have been leveled against the absolute status of the second law of thermodynamics. During this period, roughly 40 papers have appeared in the general literature [e.g., 1- 20], representing more than a dozen distinct challenges; the publication rate is increasing. Recently, for the first time, a major scientic press has commissioned a monograph on the the subject and a first international conference has been convened to examine these challenges. (..) The genealogy of the Maxwell demon thus split into those that relied on sentient processes (e.g., intelligent active measurement, calculation, or microscopic manipulation), and those that did not. The former line has largely died out owing to advances in information theory [26], but the latter survived and now poses the most serious threat to the absolute status of the second law.

Future historians of science may well put the second “law” in the same category as “heavier-than-air flying machines are impossible”. An expression of contemporary scientific prejudice and lack of technological sophistication, not an eternal law of nature.

6. The Skeptic claims that the claimed effect contradicts the “laws of nature” (and therefore has to be wrong, since the Skeptic and the scientific community he presumes to represent have of course already complete knowledge of the laws of nature).

For example, in a personal note published on James Randi’s Website, Robert Park makes the following statement about the “Motionless Electromagnetic Generator”, a claimed free energy device:

I’ve been following the MEG claim since Patent 6,362,718 was issued in the spring (What’s New 4 Apr 02). The claim, of course, is preposterous. It is a clear violation of the conservation of energy.

But Park is only demolishing a straw man. The first law of thermodynamics states that the energy of a closed system is conserved. But the inventors of the MEG claim that their device takes energy from the zero-point field of the vacuum, thereby conserving the energy of the total system (which in this case would be the MEG and the surrounding vacuum). Whether it can actually do that is an open question. But the existence of the Casimir force proves that in principle such extraction of energy from the vacuum is possible (even though the energy that can be gained from the Casimir force between two plates is negligible). Therefore, one cannot dismiss claims for free energy devices such as the MEG on a priori grounds of energy conservation. Since Park is a physicists, he could not possibly be unaware of this. By stating that the claimed invention contradicts the law of energy conservation, he intentionally misrepresents the claims of the MEG inventors. They do not claim to have found a way around the first law; they merely claim to have accessed a source of energy not previously accessible to human technology.

7. The Skeptic believes in scientific mob rule. “In Science, the Majority Consensus is Always Right”.

The unfortunate reality is that there is a complex sociology of science. Scientific truth is frequenly not determined by right or wrong, but by ego, prestige, authority of claimants, conflicts of interests and economic agendas. Scientists who propose research that threatens the viability of basic theories on which authorities in the field have built their careers, and governments and corporations have bet lots of money will find themselves out of a job very soon. The list of of great scientists who became scientific outcasts after they published research that contradicts establishment dogma is long, and includes such names as Peter Duesberg, Brian Josephson, Jacques Benveniste, and of course Professors Pons and Fleischmann.

© 2004. This text may be freely copied and/or reposted as long as it is not changed and reproduced in its entirety.

A lot of hot air has been expended on homeopathy’s apparent inability to “prove” itself in clinical trials. Many people, many of whom call themselves scientists, seem only to need this fact, plus the therapy’s apparent implausibility, to jump to the conclusion the whole thing is nonsense on stilts, and work themselves up into lathers of righteous indignation about the fact that it continues to be practiced. I’ve gone into this a couple of times in the comments to posts on this blog, but this question really does deserve detailed examination, because the issue is not at all as simple as it might seem.

The presupposition of clinical trials is that there is a stable, locally active cause that is only active in the treatment group, irrespective of blinding and the circumstances of the trial or any changed clinical context as a result of the trial. In plain English, this means that the whole basis of clinical trials is predicated on the assumption that the bulk of the treatment effect resides in the physical substance that’s being trialed. It’s a localist hypothesis, proceeding — in homeopathy’s case — from the following logic:

The localist hypothesis of homeopathy is intuitively appealing, simple, and the most straightforward. It presupposes that:
1. Homeopathy works; we know this from clinical practice. This is conceded as a heuristic starting point.
2. If homeopathy works, it cannot be molecules that are the active principle, because at high homeopathic potencies they are statistically too few to be biologically active. This is a logical extrapolation from the known body of biomolecular knowledge.
3. If molecules are not the active principle, it must be something else that is fixed to or in the remedy, and hidden from ordinary analysis for lack of sufficiently sensitive instruments, or theory, or both.
It is a localist hypothesis because it presumes the active principle has to be a local resident to the remedy. It is construed as residing in the material substance.

Walach, H. Reinventing the Wheel Will Not Make It Rounder: Controlled Trials of Homeopathy Reconsidered. Journal of Alternative and Complementary Medicine.,Vol 9, No 1, 2003, pp7–13

There’s actually no logical reason why this should be so. Think about it. The rationale behind the assumption is based on a habitual way of thinking arising from acclimation to the parameters of the pharmaceutical model of intervention. Pharmaceutical interventions are based on the mid to high dose ranges of the dose-response curve. Even allowing for the Arndt-Schulz law (now renamed hormesis), homeopathic remedies are clearly right off that scale. Why should we then presume they behave in the same way? And can be tested accordingly?

Experienced homeopathic researchers have made exactly this point:

Imagine the following situation as depicted in the figure: Let there be two treatments x and y for the same condition, say chronic pain. Let there be two placebo controlled RCTs with comparable patient populations. In every one of these trials we will have measurement artefacts caused by unreliability of measures; let them be equal in all groups. In every one of these trials, we will also have regression to the mean as a statistical artefact and as a result of the natural course of the disease studied; some patients will improve regardless of the treatment applied. Then there will be nonspecific treatment effects: Patients expect to get better when treated, especially in a trial. Hope will work against the general demoralization caused by disease. The attention of doctors and nurses within the context of a trial and perhaps the special attention paid to patients within the context of a particular CAM intervention such as homeopathy, healing, or acupuncture, will also contribute to the nonspecific part of improvement. Let us not forget that a treatment that can help patients to understand their suffering by providing an explanation, a common explanatory myth, is a therapeutic factor, too (Frank, 1989). And then there will be specific factors of treatment. Let us assume that treatment y is specifically effective. Its specific efficacy will be 20%, which, in a trial that is adequately powered, will be significant. Thus, everybody will conclude: Treatment y is an effective treatment for chronic pain. Treatment x only has 10% specific efficacy and let us assume that studies of treatment x are generally underpowered to find this effect. Everybody will conclude: Treatment x is an ineffective treatment for chronic pain. What usually is overlooked is the fact that the nonspecific treatment effects of treatment x are much larger. In the thought experiment, I have chosen them to be 30% for treatment x. For treatment y, they would only be 5%. In such a case treatment x, although overall much more powerful with 70% of patients potentially benefitting from it by virtue of its strong nonspecific effects, would be neglected in favor of treatment y, with 55% of patients benefitting from it, because y has a stronger specific treatment effect.

I maintain that this situation is frequently true for CAM therapies. Studies are often underpowered, eg., for acupuncture, and thus potential specific effects are overlooked. The conclusion of reviewers and the educated public then is the verdict “inconclusive evidence” (Ezzo, et al., 2001), and the political consequence, as just happened in Germany, is the decision to not include acupuncture in the scheme for public reimbursement, because the evidence for specific efficacy is inconclusive (Bundesausschuss Ärzte und Krankenkassen, 2001). However, nobody pays attention to the fact that perhaps the magnitude of nonspecific effects makes a treatment effective and not the specific effects. An even more complicated situation can arise when the circumstances of a trial, such as blinding and changing the natural flow of patient–doctor interaction and treatment sequences, change the context of a treatment dramatically and thus alter the potential nonspecific effects in a detrimental way. This can happen in blinded trials of homeopathy, in which insecurity arises from the blinding of doctors, and also in trials of acupuncture, when blinding procedures make it necessary that the doctor who is taking the case and making the assessment is different from the person who is administering the treatment. In all such cases, trials may alter the context of a treatment and thus diminish potent nonspecific factors and thereby underestimate effectiveness.

Walach, H. The Efficacy Paradox in Randomized Controlled Trials of CAM and Elsewhere: Beware of the Placebo Trap. Journal of Alternative and Complementary Medicine, Vol 7, No 3, 2001, pp213-218

This thought experiment of Walach’s models the experience with testing homeopathy and other CAM therapies very well. Data from clinical studies consistently suggest a positive treatment effect of around 70% for homeopathy, yet repeated attempts to replicate these results in controlled trials have failed. It therefore becomes necessary to explain this discrepancy, rather than assume clinical results, especially in such large cohort studies as have been undertaken, can be put down to such notions as “bias”, “regression to mean” and “placebo response”.

It still remains, as Kate Chatfield has remarked:

… if homeopaths can facilitate a placebo-induced healing response in over 70% of people who visit them, many of whom have previously not been helped by various types of allopathic intervention, then surely homeopaths should be highly revered and re-labelled ‘miracle workers’.

Chatfield, K. In Pursuit of Evidence.

As I’ve argued repeatedly, the phenomenon that is homeopathy can’t simply be written off. We need to re-examine the assumptions underlying trial design and look at other ways of satisfactorily evidencing efficacy.

While musing on the nature of evidence, and what is or is not considered ‘evidence’ in all the various contexts evidence is used, I came across this excellent rapid response in the BMJ back in 2004 by Clifford Miller, “a practising English lawyer, graduate in physics and a sometime examining lecturer on law, standards and ethics (particularly, the law of evidence) to Masters student technologists at the Imperial College of Science Technology and Medicine”.

He writes, in the context of the imputed association between MMR vaccination and the onset of autistic states, on “The Unreliability of Scientific Papers as Evidence”:

“Reliable evidence is that which is authentic, accurate and complete. In short, scientific evidence is incomplete if used for purposes outside the strict confines of science because it fails to take account of evidence of lay witnesses of the facts and is hence only applicable to the narrow and specific confines of scientific enquiry and not the broader ones found in other fields of human endeavour.

“Examples in point include the parental evidence of symptoms in the MMR cases or that of Gulf War veterans about their symptoms. A court (or the Legal Services Commission in the case of MMR) in contrast, ought to take that oral evidence into account for the very reasons science dismisses it. The point, unfortunately is not as well taken by our legal system as it might be. We have seen this recently with the Legal Services Commission in the MMR cases and in the cases of Gulf War veterans.

“Science treats evidence of lay witnesses of fact as inadmissible (as ‘anecdotal’ only) for reasons which are inapplicable in Court, but science does so for two main reasons. The higher scientific standard of proof (in effect, irrefutability) only admits evidence which can be tested scientifically for reliability. Oral witness evidence is discounted by medical science because medical scientific method does not currently have or recognise a mechanism for testing oral evidence to the scientific standard and so, for the sake of rigour, excludes it.

“Neither of these propositions apply in Court. Evidence of the direct witness of the fact, whether oral, or more frequently now, by way of written statement, is always admissible and is, in fact, the keystone of the trial system of evidence and the primary source of information a court uses to make decisions of fact. The Court has and applies its own mechanisms for testing witness evidence (eg. cross-examination). Further, the Court applies a far lower standard of proof, namely a balance of probability and not the unnecessarily high one of irrefutability applied by science.

[...]

“Governments also take advantage of the confusion and often use the term ‘evidence’ interchangeably with ‘proof’ when dismissing evidence they choose not to agree with or set unreasonably high standards of proof for the kind of decision required. The press and public alike are continually hoodwinked by this approach.

“In law ‘evidence’ is nothing more than information. It is information which one party proposes in support of, or to undermine, a disputed proposition. ‘Proof’, however, depends upon the decision-making process concerned. For the public interest, the standard of proof is sometimes based on risk and sometimes on other factors. In civil courts it is ‘balance of probability’. In criminal it is ‘beyond reasonable doubt’. And science requires irrefutable proof: a remarkably high standard.

“It is a fundamental error to apply the wrong standard of proof to the decision making process concerned and yet it seems to happen regularly.

“For issues of public safety, such as medicines like MMR or vaccines in the Gulf War, or the BSE crisis, the risk standard ought to be applied.

“However, instead, we, the public, are told frequently by officials in government there is no scientific evidence of a causal link between one thing and another. Whereas, often evidence to the contrary does in fact exist, it is not evidence that the officialdom concerned may either choose to or sometimes be at liberty to accept as proof of the issue. This is much the same for the BSE crisis with the government as it was for the Courts in relation to Gulf War syndrome or the Legal Services Commission for MMR. Whereas in the case of courts, the court has to rely on the expert evidence presented, in the case of public health officials like the Chief Medical Officer, he is in a position to assess the reliability himself, with the aid of his own experts if necessary. However, in the latter case, the risk standard of proof ought to be applied to decision making in the public interest rather than the scientific standard, which is only applicable to proof in science.

“In the scientific context, the only answer to a scientific issue that scientific journals should involve themselves in is a scientific one and they should only trouble themselves with the scientific standard of proof. If MMR did not cause autism, or vaccines in the Gulf War did not cause other problems, then it is for scientific journals to publish irrefutable scientific proof of what ails the 1000 or so children and the numerous afflicted Gulf War veterans.

“The current political debates about these kinds of issues are ones science could answer, if only the scientists got on with it and stopped playing politics.

“It is, for example, no answer to Wakefield to claim there is no scientific evidence of a link between MMR and autism. That just shows science has not found one that it can accept as proven to its very high standard of proof. It does not prove there are none, nor that there is no proof to other more realistic and practical standards for day-to-day decision making. It also leaves the public confused and distrustful of science.

“Buried in the MMR debate and little known to the general public is formal confirmation of a link between immunisation and the so-called allergy epidemics in the developed world. According to the US National Academies’ Institute of Medicine (IoM) Immunization Safety Review Committee (1), for at least two years it has been known that current vaccination programmes can expose children to risk of various problems ranging from allergy to infection. The IoM have also confirmed (2) that reasonable theories exist to explain how too many immunizations can overwhelm an infant’s immune system.

“A clear indication of the possibility of the existence of a causal connection between vaccination and the emergence of the various allergy and other issues over the last 20 years is the contemporaneous substantial increase in vaccinations as reported by the IoM (3). This shows an increase from 4 vaccinations per child in 1980 to up to 20 now.

“Whilst the IoM considered (4), as regards asthma in particular, and allergies in general, it had inadequate evidence to accept or reject a causal relationship, it accepted there is cause to consider that there might be a connection. Effectively, all the IoM statement amounts to is an admission by the most authoritative governmental authority in the US that they will not accept any evidence unless it provides the answer to a scientific standard of proof, and until someone produces that proof, they will not apply a risk standard, such that it is immunization as usual for children.

“The IoM’s conclusion is also not a reliable one for government to apply to the risk standard of proof because the IoM rely upon the scientific standard of proof and that is the wrong standard to apply for a decision based on risk. Irrefutability is too high a hurdle. Similarly, parents taking practical day-to-day decisions risk their child’s health if they wait for scientific proof, because proof to such a standard also takes too long to be produced.

Quite. Yet note that the MMR vaccination was foisted upon the population at large on the basis of those same scientific standards of evidence. So scientific standards of evidence conclude the vaccine has “proven” efficacy and “safety”, despite the fact that trials of the vaccine are limited and have no long-term follow-up. Scientific standards of evidence also conclude that there’s no adequate association between MMR vaccination and autism, despite the fact that documentary evidence acceptable as prime evidence in a court of law presents compelling evidence to the contrary.

What’s wrong with this picture? The aim of science is to arrive at an explanation and understanding of observed phenomena and their causal mechanisms that is, above all, utterly faithful to those observed phenomena. Not politics, not commercial expediency, not pre-existing theories or biases. The observed phenomena. Full stop, end of story. Consequently any hypothesis that meets those conditions should be acceptable to anyone observing the phenomena under consideration because the degree of fit will be such as to render any alternative explanation superfluous and irrelevant.

But when there is such substantial variance between the observed phenomena according to day-to-day standards of evidence and the preferred ’scientific’ hypothesis, then something has to have gone badly wrong, and the scientific process has become subverted by something other than the aim to explain the observed phenomena. If the two are not congruent, then the scientific proces has failed.

I’ve been taking a closer look at what Ben Goldacre regards as ‘good science’, given that he makes such grandiose and sweeping claims about the respective quality of research into both homeopathy and pharmaceutical interventions in both his recent Guardian article, What’s wrong with homeopathy? and the accompanying comment piece in The Lancet, Benefits and Risks of Homeopathy.

In the Guardian, he claims:

I look about 12, and I’m only a few years out of medical school. This is all good fun, but my adamant stance, that I absolutely lack any authority, is key: because this is not about one man’s opinion, and there is nothing even slightly technical or complicated about the evidence on homeopathy, or indeed anything, when it is clearly explained.

He then goes on at great length, much of it speculative, to give his reasons for why just about all homeopathic research is meaningless nonsense full of nothing but bias and conventional medical research is robust.

But thanks to a rapid response on the BMJ’s website, we learn the following from John Stone:

We can all benefit from Ben Goldacre’s wisdom. For instance, in 2004 Goldacre received the Association of British Science Writer’s (ABSW) award for “the best feature on a science subject in a national or regional newspaper (2003) [1] for an article on MMR: Never mind the facts [2]. The major sponsor of the award of £2000 was MMR manufacturer and defendent Glaxo SmithKline [1]. This has not been disclosed in many Guardian articles mentioning MMR, nor was it recently in Goldacre’s BMJ piece MMR, the scare stories are back [3].

I also wonder whether this is one of the best examples of Goldacre’s work. Goldacre cited four studies three of which were subsequently reviewed by Cochrane 2005 [4]. Of these Cochrane stated:

“The study demonstrates the difficulties of drawing inferences in the absence of a non-exposed population or a clearly defined causal hypothesis”. (Re: Taylor 1999)

“The number and possible impact of biases in this study was so high that interpretation of the results is impossible”. (Re: Fombonne 2001)

“The interpretation of the study by Madsen was made difficult by the unequal length of follow up for younger cohort members as well as the use of the date of diagnosis rather than onset of symptoms of autism”. (Re: Madsen 2002)

The fourth study mentioned was the most remarkable of all, the Peltola letter to the Lancet of May 1998 which recorded no cases of autism or inflammatory bowel disease following 3 million applications of MMR in Finland simply because they were not in the follow up criteria of the larger study [5].

Moreover, there was a complex of funding issues unrelated by Goldacre. The Peltola study received funding from MMR defendent Merck [5]. The department of Elizabeth Miller who contributed to the Taylor study [6] benefited from funding by MMR defendents SmithKline Beecham and Aventis Pasteur [7]. At least one MMR study authored by her and Brent Taylor received funding from SmithKline Beecham [8]. The Fombonne study disclosed no interests [9] but according to a later study [10]:

“In the United Kingdom, Dr Fombonne has provided advice on the epidemiology and clinical aspects of autism to scientists advising parents, to vaccine manufacturers, and to several government committees between 1998 and 2001. Since June 2004, Dr Fombonne has been an expert witness for vaccine manufacturers in US thimerosal litigation. None of his research has ever been funded by the industry.”

So you can’t be too careful!

[1] ABSW Science Writers Award 2003

[2] Ben Goldacre, Never mind the facts, Guardian 11 December 2003

[3] Ben Goldacre, MMR, the scare stories are back, BMJ 21 July 2007

[4] V Demicheli, T Jefferson, A Rivetti, D Price,[Review] ‘Vaccines for measles, mumps and rubella in children’, Cochrane (Wiley 2005)

[5] Heikki Peltola, Annamari Patja, Pauli Leinikki, Martti Valle, Irja Davidkin, Mikho Paunio, No evidence for measles, mumps, and rubella vaccine-associated inflammatory bowel disease or autism in a 14-year prospective study, Lancet vol 351, May 1998, p. 1327-8

[6] Brent Taylor, Elizabeth Miller, C Paddy Farrington, Maria- Christina Petropoulos, Isabelle Favot-Mayaud, Jun Li, Pauline A Waight, Autism and measles, mumps, and rubella vaccine: no epidemiological evidence for a causal association, Lancet vol 135, 12 June 1999.

[7] According to Geier M and Geier D (P3R to PEDIATRICS,’ Thimerosal does not belong in vaccines’ 8 September 2004) this funding was disclosed by Elizabeth Miller to the Committee on Safety of Medicines in 2002, and this was not denied by Dr Miller in her response

[8] E Miller, P Waight, C P Farrington, N Andrews, J Stowe, B Taylor, Idiopathic thrombocytopenic purpura and MMR vaccine. vol. 84 p.227-9 March 2001

[9] Eric Fombonne, FRCPsych and Suniti Chakrabarti, FRCPCH, No Evidence for A New Variant of Measles-Mumps-Rubella-Induced Autism, PEDIATRICS Vol. 108 No. 4 October 2001, p. e58

[10] Eric Fombonne, MD, Rita Zakarian, ME, Andrew Bennett, PhD, CPsych, Linyan Meng, MSc and Diane McLean-Heywood, MA, Pervasive Developmental Disorders in Montreal, Quebec, Canada: Prevalence and Links With Immunizations, Published online July 3, 2006 PEDIATRICS Vol. 118 No. 1 July 2006, pp. e139-e150 (doi:10.1542/peds.2005- 2993)

Competing interests: Autistic son

Curioser and curioser …

And yes, this appears to be very much about one man’s opinion.

Goldacre’s defence of MMR vaccination should also be viewed in the context of a study recently undertaken in the US. The new survey indicates a strong correlation between rates of neurological disorders, such as ADHD and autism, and childhood vaccinations.

The survey, commissioned by Generation Rescue, compared vaccinated and unvaccinated children in nine counties in Oregon and California. Among more than 9,000 boys age 4-17, the survey found vaccinated boys were two and a half times (155%) more likely to have neurological disorders compared to their unvaccinated peers. Vaccinated boys were 224% more likely to have Attention Deficit Hyperactivity Disorder (ADHD), and 61% more likely to have autism.

For older vaccinated boys in the 11-17 age bracket, the results were even more pronounced. Vaccinated boys were 158% more likely to have a neurological disorder, 317% more likely to have ADHD, and 112% more likely to have autism.

So now we have another prominent ’scientist’ joining the chorus. Sir David King, the government’s chief scientific advisor, speaking to MPs on the innovation, universities and skills select committee about his role, laid into sections of the media (for their “campaigns” against GM foods and the MMR vaccine) and the Department of Health over its decision to allow homeopathic remedies to be licensed by the Medicines and Healthcare Products Regulatory Agency, the public body that licenses drugs.

“How can you have homeopathic medicines labelled by a department which is driven by science?” said King. “There is not one jot of evidence supporting the notion that homeopathic medicines are of any assistance whatsoever.”

(Reports in the Guardian, the Daily Telegraph, the Daily Mail)

Make no mistake, this “not one jot of evidence” is nothing more than a mantra which those continually repeating it seem to hope will eventually become accepted as some sort of “truth” merely through repetition. In no sense does it bear any relation to the actual state of the evidence base for homeopathy. It is, quite simply, a lie.

Evidence-based medicine is a laudable objective, but not when the whole process is being corrupted by theoretical bias as it is here to determine what is and is not “valid” evidence.

“When I asked 210 GPs to rate different forms of evidence that in practice they would want before using or recommending an unorthodox therapy their answers suggested that evidence forms a multidimensional mosaic - an ‘Evidence Profile’ 2. As Figure 1 shows, theoretical factors are seen as least important, while a systematic examination of outcome (“Experience”) is placed highest, with clinical trials next. Professional experience and patients’ views are still rated very highly, well ahead of theoretical or laboratory evidence. The nature of ‘evidence’ and ‘Evidence Based Medicine’ is evolving, seeking a balance between literature appraisal, clinical evaluation, and human caring (e.g. see http://www.cche.net/usersgui des/ebm.asp#31 ).
It is not a method to use the first of these factors to dominate the others.

“Professor Sackett opens his seminal book on Evidence Based Medicine 3 with “Evidence ­based medicine is the integration of best research evidence with clinical expertise and patient values.”"

The Evidence For Homoeopathy, D. Reilly. V8.3 Sept06

As far as homeopathy is concerned, the “best research evidence” is equivocal, but clinical evidence and patient values are most definitely not. Several clinical studies now (*), both in single conditions and in large cohort long-term evaluations of the therapy, show a high level of patient satisfaction as well as results that are comparable, and in many cases superior to, the results obtained by conventional pharmaceutical interventions.

Since Sir David King’s principal gripe about the British public’s opposition to GM foods was that Britain’s failure to adopt GM crops had cost the economy between £2bn and £4bn, it’s tempting to wonder whether his snipe at homeopathy comes from similar financial considerations on behalf of the pharmaceutical industry. This seems woefully out of touch with public opinion, which evidences more concern for the health of the biosphere and the humans inhabiting it than for the health of industry’s bank balances.

It also seems hugely disingenuous amidst recent revelations about contaminated cell lines in cancer research, almost daily reports of drugs being withdrawn for serious side effects, long-term studies showing one of the principal avenues of conventional treatment for cancer, cytotoxic chemotherapy, to have only minimal impact on survival, the ineffectiveness of antibiotic and steroid treatment for sinus infections, the fact that honey is a more effective cough medicine than the widely prescribed dextromethorphan, and so it goes on …

See also goodscience’s blog with a link to a video interview with Dr Maurice Hilleman, former chief of Merck’s vaccine division, talking about the catalogue of errors that accompanied the early development of vaccines for polio. “It was good science at the time because that was what you did …” Is there any reason to suppose, in the light of these daily revelations about the shortcomings of pharmaceutical interventions, that things are any different now?

(*) Outcome and costs of homoeopathic and conventional treatment strategies: A comparative cohort study in patients with chronic disorders. Witt, C, Keil, T, Selim, D et al. Complementary Therapies in Medicine (2005) 13, 79-86. German study concluding from an analysis of outcomes for 493 patients that patients seeking homeopathic treatment had a better outcome overall compared with patients on conventional treatment.

Homeopathic medical practice: Long-term results of a cohort study with 3981 patients. Witt, Claudia M, Lüdtke, Rainer, Baur, Roland, and Willich, Stefan N. BMC Public Health 2005, 5:115. A total of 3,981 patients were studied including 2,851 adults and 1,130 children. Ninety-seven percent of all diagnoses were chronic with an average duration of 8.8 years. Almost all patients had received conventional treatment (95%) prior to the start of this study. Disease severity decreased significantly (p<0.001) between baseline and 24 months. Disease severity and quality of life demonstrated marked and sustained improvements following homeopathic treatment period.

Homeopathic Treatment for Chronic Disease: A 6-Year, University-Hospital Outpatient Observational Study. Spence, David S and Thompson, Elizabeth A. Journal of Alternative and Complementary Medicine Volume 11, Number 5, 2005, pp. 793-798. Observational study of 6544 consecutive follow-up patients during a 6-year period in a hospital outpatient unit within an acute National Health Service (NHS) Teaching Trust in the United Kingdom. 70.7% of patients receiving homeopathic treatment reported positive health changes.

This is the second paragraph from Ben Goldacre’s recent comment piece Benefits and risks of homeopathy in The Lancet’s November 17 edition.

During the cholera epidemic in the 19th century, death rates at the London Homoeopathic Hospital were three times lower than those at the Middlesex Hospital. (6) The reason for homoeopathy’s success in this epidemic is even more interesting than the placebo effect. At the time, nobody could treat cholera, and while medical treatments such as blood-letting were actively harmful, the homoeopaths’ treatments were at least inert.

(6) Hempel S. The medical detective. London, UK: Granta Books, 2006

Notice how Goldacre doesn’t give us the actual figures. (And this is The Lancet, not the Guardian.) The actual mortality percentages would allow readers to make some kind of sensible judgement about his conjecture that medical treatments exacerbated mortality while homeopathic treatment equated to no treatment at all.

Here they are:

In the London Cholera epidemic of 1854, of the 61 cases of cholera treated [at the London Homeopathic Hospital], 10 died, a percentage of 16.4; of the 331 cases of choleraic and simple diarrhoea treated, 1 died. The neighbouring Middlesex Hospital received 231 cases of cholera and 47 cases of choleraic diarrhea. Of the cholera patients treated 123 died, a fatality rate of 53.2 per cent., among the victims being one of the nurses.

Morrell, P and Cazelet, S. The History of the London Homeopathic Hospital

From a quick trawl through various literature on the subject, it would appear untreated cholera has a mortality rate of anywhere between 30-80%. Typically it seems to be around 50-60% and can kill within a matter of hours.

16.4% mortality against 50-60% mortality for untreated cholera would not appear to be consistent with the idea that homeopathic treatment was inert. 16.4% mortality against even 30% mortality for untreated cholera would not appear to be consistent with the idea that homeopathic treatment was inert. At the time, the London Homeopathic Hospital was at Golden Street, close to the epicentre of the outbreak.

The figures for homeopathic treatment were so striking to the medical establishment of the time, that they suppressed them.

Now, a circular was addressed by the President of the Board of Health to various Metropolitan hospitals and to qualified practitioners, requesting returns of cholera cases, with details of the circumstances, treatment and results. The object was to determine by comparison, for the public good, what treatment experience showed to be the best for the new plague. Returns were sent in from the London Homœopathic Hospital, giving the names and addresses of the patients treated, the symptoms, remedies, and result in each case, and a summary of those results. This was not a question of theory, or of any particular school; it was a question of facts and statistics affecting the public health. But the report of the Board of Health was presented to Parliament without the slightest reference to the London Homœopathic Hospital or to the brilliant results which its physicians had achieved by undaunted self-sacrifice in a time of great public calamity. Complaint was, of course, made to the Board of Health and duly reterred to its Medical Committee, with the result that the Board received from the committee a resolution, which, for ingenuity of disingenuousness and illiberality, can hardly ever have been equalled. It was this :

“That by introducing the returns of homœopathic practitioners they (the Treatment Committee) would not only compromise the value and utility of their averages of Cure, as deduced from the operation of known remedies, but they would give an unjustifiable sanction to an empirical practice, alike opposed to the maintenance of truth and to the progress of science.”

In the first place, the remedies “unknown” to the Treatment Committee were such as camphor, copper, hellebore, arsenic, and other drugs well known to medicine. In the second place, it was their bounden duty to “compromise” the averages of old methods by more successful new methods in their search for the best results. Thirdly, the interference with empirical practice was no part of the statistical duty before them. And lastly, the “progress of science” was de facto obstructed by their refusal to “compromise” their averages by a factor which contained the very object of their search. The perversity was too plain, and Lord Robert Grosvenor (afterwards Lord Ebury) moved on May 17, 1855, in the House of Commons for “Copies of Letters addressed to the General Board of Health complaining of the omission of any notice of certain returns in relation to the treatment of cholera and correspondence between the President of the Board and the Medical Council, with copies of the returns which have been rejected by the Medical Council.” The House of Commons, which was more anxious for the “progress of science” and the “value and utility of averages” than for “the operation of known remedies,” to say nothing of its great duty to the people it represented, forthwith ordered a special return of the ignored homœopathic statistics, which was in due course made by the Board of Health, and these returns were ordered by the House to be printed on May 21, 1855. They remain among Parliamentary Papers to this day, a standing monument alike of the success of the new policy and of the obscurantism of the old.

Morrell, P and Cazelet, S. The History of the London Homeopathic Hospital

These results were not an isolated fluke. According to Julian Winston, homeopathic historian, who made an extensive search of the literature:

“When Cholera finally struck Europe in 1831 the mortality rate (under conventional treatment) was between 40% (Imperial Council of Russia) to 80% (Osler’s Practice of Medicine). Out of five people who contracted Cholera, two to four of them died under regular treatment.

“Dr Quin, in London, reported the mortality in the ten homeopathic hospitals in 1831-32 as 9%; Dr Roth, physician to the king of Bavaria, reported that under homeopathic care the mortality was 7%; Admiral Mordoinow of the Imperial Russian Council reported 10% mortality under homeopathy; and Dr Wild, Allopathic editor of Dublin Quarterly Journal, reported in Austria, the Allopathic mortality was 66% and the homeopathic mortality was 33% “and on account of this extraordinary result, the law interdicting the practice of Homeopathy in Austria was repealed.”

Winston, Julian. Some history of the treatment of epidemics with Homeopathy. 2001. Unpublished work

This is the first paragraph from Ben Goldacre’s recent comment piece Benefits and risks of homeopathy in The Lancet’s November 17 edition.

Five large meta-analyses of homoeopathy trials have been done. All have had the same result: after excluding methodologically inadequate trials and accounting for publication bias, homoeopathy produced no statistically significant benefit over placebo. 1–5

(1) Kleijnen J, Knipschild P, ter Riet G. Clinical trials of homoeopathy. BMJ 1991; 302: 316–23.
(2) Boissel JP, Cucherat M, Haugh M, Gauthier E. Critical literature review on the effectiveness of homoeopathy: overview of data from homoeopathic medicine trials. Brussels, Belgium: Homoeopathic Medicine Research Group. Report to the European Commission. 1996: 195–210.
(3) Linde K, Melchart D. Randomized controlled trials of individualized homeopathy: a state-of-the-art review. J Alter Complement Med 1998; 4: 371–88.
(4) Cucherat M, Haugh MC, Gooch M, Boissel JP. Evidence of clinical efficacy of homeopathy: a meta-analysis of clinical trials. Eur J Clin Pharmacol 2000; 56: 27–33
(5) Shang A, Huwiler-Müntener K, Nartey L, et al. Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 2005; 366: 726–32

Note that Goldacre omits the Linde et al meta-analysis published in The Lancet in 1997 (6) from his listed studies.

Below are comments and conclusions from each of these studies. Remember, Goldacre is saying that they each support his assertion that homeopathy has no statistically significant benefit over placebo.

(1)

The 1991 study by Jos Kleijnen, Paul Knipschild and Gerben ter Riet assessed the methodological quality of 107 controlled trials in 96 published reports found after an extensive search. Trials were scored using a list of predefined criteria of good methodology, and the outcome of the trials was interpreted in relation to their quality. In 14 trials some form of classical homoeopathy was tested and in 58 trials the same single homoeopathic treatment was given to patients with comparable conventional diagnoses. Combinations of several homoeopathic treatments were tested in 26 trials; isopathy was tested in nine trials. Most trials seemed to be of very low quality, but there were many exceptions. The results showed a positive trend regardless of the quality of the trial or the variety of homoeopathy used. Overall, of the 105 trials with interpretable results, 81 trials indicated positive results whereas in 24 trials no positive effects of homoeopathy were found.Within the discussion of their findings, the authors state:

“The amount of positive evidence even among the best studies came as a surprise to us. Based on this evidence we would be ready to accept that homoeopathy can be efficacious, if only the mechanism of action were more plausible.”

And go on to say:

“The way in which the belief of people changes after the presentation of empirical evidence depends on their prior beliefs and on the quality of the evidence. Critical people who did not believe in the efficacy of homoeopathy before reading the evidence presented here probably will still not be convinced; people who were more ambivalent in advance will perhaps have a more optimistic view now, whereas people who already believed in the efficacy of homoeopathy might at this moment be almost certain that homoeopathy works.”

They concluded:

“At the moment the evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias. This indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.”

(2)

I don’t have a copy of this paper. This comment on it is extracted from a 2002 Italian literature review (Cornelli, Prof Umberto et al):

“These experts identified 377 clinical trials, short-listed 220, and reviewed 184. Detailed research lasting several months was conducted on the best trials, to evaluate their scientific value.The conclusions researched by the Advisory Group are unequivocal: the number of significant results cannot be attributed to chance. The analysis provided a random hypothesis value of p < 0.001.The Advisory Group remained very cautious, but expressly stated: “The null hypothesis that homeopathy has no effect can be rejected with certainty; in other words, in at least one of the studies examined the patients treated with the homeopathic remedy received benefits compared with the control patients who received the placebo”.”

(3)

Klaus Linde and Dieter Melchart’s paper on Randomized Controlled Trials of Individualized Homeopathy was a limited study designed to test a subset of homeopathic trials. Because of its small size, it’s consequently not a particularly appropriate study to cite in support of generalised conclusions about the therapy as a whole, even if it does test trials that attempt to adhere more closely to homeopathy as it’s practiced.

“Randomized or quasirandomized controlled clinical trials comparing an individualized homeopathic treatment strategy with placebo, no treatment, or another treatment were eligible. Information on patients, methods, interventions, outcomes, and results was extracted in a standardized manner and quality was assessed using a checklist and two scoring systems. Trials providing sufficient data were pooled in a quantitative meta-analysis.”

Results:

“A total of 32 trials (28 placebo-controlled, 2 comparing homeopathy and another treatment, 2 comparing both) involving a total of 1778 patients met the inclusion criteria. The methodological quality of the trials was highly variable. In the 19 placebo-controlled trials providing sufficient data for meta-analysis, individualized homeopathy was significantly more effective than placebo (pooled rate ratio 1.62, 95% confidence interval 1.17 to 2.23), but when the analysis was restricted to the methodologically best trials no significant effect was seen.”

Conclusions:

“The results of the available randomized trials suggest that individualized homeopathy has an effect over placebo. The evidence, however, is not convincing because of methodological shortcomings and inconsistencies. Future research should focus on replication of existing promising studies. New randomized studies should be preceded by pilot studies.”

(4)

This paper was financed by the European Commission and undertaken by the Homoeopathic Medicine Research Group as in (2) above.

“The selection criteria were randomised, controlled trials in which the efficacy of homeopathic treatment was assessed relative to placebo in patients using clinical or surrogate endpoints. Prevention trials or those evaluating only biological e€ffects were excluded. One hundred and eighteen randomised trials were identified and evaluated for inclusion. Sixteen trials, representing 17 comparisons and including a total of 2617 evaluated patients, fulfilled the inclusion criteria … The reasons for exclusion of the remaining 102 trials were primary outcome not clearly defined (92, 90%) and methodological defects (10, 10%).”

Comment:

“The signficant combined P value obtained in the main analysis does not imply that the homeopathic treatments were efficacious in all the pooled comparisons. This result provides evidence that in at least one trial the homeopathic treatment was more efficacious than placebo. In other words, more trials had a positive result than would be expected due to chance alone.“

Publication bias:

“Although we cannot exclude the possibly that the results of the meta-analysis are a€ffected by publication bias, the results of the sensitivity analysis suggest that this is unlikely.”

Conclusion:

“From the available evidence, it is likely that among the tested homeopathic treatments tested at least one shows an added e€ffect relative to placebo. The meta-analysis method used does not allow any conclusion on what homeopathic treatment is e€ffective in which diagnosis or against which symptoms. It is of no more practical value than to answer yes to the question “are homeopathic treatments e€ffective?” without specifying which drug?, which dose or regimen? and against which disease?However, the strength of the evidence for this conclusion remains low because of the overall low quality of the trial designs and reporting and the limitations of the meta-analysis approach used.
[...]
It is clear that the strength of available evidence is insufficient to conclude that homeopathy is clinically e€ffective; however, homeopathy can and should be assessed using the same methodology used for allopathy. More well-designed and well-run clinical trials, including many hundreds of patients, are needed before definitive conclusions can be drawn regarding the clinical efficacy of homeopathic treatments.”

(5)

The Shang et al meta-analysis published in The Lancet in August 2005 and trumpeted in a blaze of nationwide publicity as “the end of homeopathy” really deserves an entry all to itself. It attracted widespread condemnation and its methodological weaknesses, coupled with its blatant violations of transparency, have led many reviewers to state emphatically that it should never have passed peer review.

“When orthodox scientists, statisticians, molecular chemists, clinicians, and mathematicians, and rigorous, scientifically trained, academic clinical homeopaths begin corresponding in response to the publication of a paper in a learned journal, to draw attention to a serious scientific error, quite apart from its associated moral and ethical implications, and when letter after letter, quietly reasoned, and objectively critical of the original publication, is rejected by the initiating journal, it is surely time to reflect very deeply on what might be taking place and to ask “why?””

Jobst, K A. Homeopathy, Hahnemann, and The Lancet 250 Years On: A Case of the Emperor’s New Clothes? Journal of Alternative & Complementary Medicine. Volume 11, Number 5, 2005, pp. 751–754

Of the total 220 matched trials (110 for each therapy), the authors identified 21 “high-quality” homeopathic studies, and 9 “high-quality” conventional studies. No comparative analysis of this subset of trials was presented. The authors then proceeded to further select the small subset of purportedly larger and higher methodological quality trials (8 homeopathy trials and 6 conventional medicine trials) from which the paper’s conclusion that homeopathy is no better than placebo is drawn, but failed to describe the weighting of their selection attributes (size and methodological quality). They didn’t explain how they chose the particular cut off point that they used to select the final 14 trials, neither did they identify which trials these were, neither did they provide any information on which to assess whether those trails were still matched. Initial requests by other researchers to identify these trials were refused repeatedly and data finally permitting their identification was not made available until several months after.

Most telling of all perhaps is the following from the Discussion section of the paper:

“We assumed that the effects observed in placebo-controlled trials of homoeopathy could be explained by a combination of methodological deficiencies and biased reporting. Conversely, we postulated that the same biases could not explain the effects observed in comparable placebo-controlled trials of conventional medicine.”

This a priori assumption means that essentially the study can do nothing but produce a null result. Having filtered for methodological deficiency, the assumption is that any positive effect remaining must be bias. By those standards, the only course of action left is to remove all the trials with any positive effect and leave the ones that show no benefit.

The authors do at least admit that it they’d chosen a different set of 8 homeopathy trials, the results would have been quite different:

“For example, for the eight trials of homoeopathic remedies in acute infections of the upper respiratory tract that were included in our sample, the pooled effect indicated a substantial beneficial effect (odds ratio 0·36 [95% CI 0·26–0·50]) and there was neither convincing evidence of funnel-plot asymmetry nor evidence that the effect differed between the trial classified as of higher reported quality and the remaining trials. Such sensitivity analyses might suggest that there is robust evidence that the treatment under investigation works. However, the biases that are prevalent in these publications, as shown by our study, might promote the conclusion that the results cannot be trusted.”

The study’s designer, Matthias Egger, is well known for his anti-homeopathy stance. His assumption that any positive effect from homeopathy, after allowing for methodological deficiency, is due to publication bias merely highlights the extent of his own bias.

(6)

Linde K, Clausius N, Ramirez G, Melchart D, Eitel F, Hedges LV, Jonas WB. Are the clinical effects of homeopathy placebo effects? A meta-analysis of placebo-controlled trials. The Lancet. 1997 Sep 20;350(9081):834-43.This is the meta-analysis which Goldacre omitted to mention.

“Double-blind and/or randomised placebo-controlled trials of clinical conditions were considered. Our review of 186 trials identified 119 that met the inclusion criteria. 89 had adequate data for meta-analysis, and two sets of trial were used to assess reproducibility.”

Findings:

“The combined odds ratio for the 89 studies entered into the main meta-analysis was 2·45 (95% CI 2·05, 2·93) in favour of homoeopathy. The odds ratio for the 26 good-quality studies was 1·66 (1·33, 2·08), and that corrected for publication bias was 1·78 (1·03, 3·10).”

Implications:

“We believe that a serious effort to research homoeopathy is clearly warranted despite its implausibility.”

Conclusion:

“The results of our meta-analysis are not compatible with the hypothesis that the clinical effects of homoeopathy are completely due to placebo. However, we found insufficient evidence from these studies that homoeopathy is clearly efficacious for any single clinical condition.”

The evidence from these studies is not conclusive either way. The general consensus is that there seems to be an effect greater than placebo, but it’s not very strong in trials of higher quality and that more studies, and more studies of different kinds testing the therapy in different ways, are needed. This is a long long way from supporting Goldacre’s null hypothesis.

For more comment, see this annex to The European Committee for Homeopathy’s press release on The Lancet’s November 17 issue.

OK. Enough is enough. Everyone’s entitled to their own opinions, but when they start claiming their opinions are ’scientific’, write comments in learned journals quoting published papers in support of their arguments which don’t actually support their arguments at all, misrepresent statistics and generally twist things so far around their little fingers it defies belief, then if you’re a scientist, a homeopath, both or neither, it’s time to put the record straight.

Dr Ben Goldacre, junior doctor and journalist for the UK Guardian, has been adopting a high profile of late. His attacks on alternative therapies, and homeopathy in particular, have reached such a fever pitch it resembles more of a witch-hunt than a scientific debate. Of course all spindoctors know that if you repeat something often enough, sooner or later people start to believe it, whether it’s true or not. Goldacre’s writings, featuring a persuasive mix of jocular sarcasm, apparent scientific plausibility and fearmongering, would have you believe there’s now scientific consensus from unanimous evidence proving that homeopathy is no more than placebo. It’s one thing to publish that in the popular press, another entirely when it appears in The Lancet.

There is, in fact, no factual basis for these assertions. The evidence Goldacre cites doesn’t back up his claims and the ’science’ he claims to champion is little more than opinion and spin. Of course I don’t expect you to believe me just because I said so, or even because I include citations (1) of papers published in peer reviewed journals in my footnotes. I’m not even asking you to believe me. I’m asking you to look at the actual scientific proof itself, and to do so with a thorough and critical eye.

For copyright reasons I can’t publish that in its entirety here, but I’ll quote what portions I can from these papers to give the gist of what they’re about in a way that’s hopefully as intelligible to a non-expert as it is to an expert. I’ll present evidence to show that in some instances the peer review process has failed, resulting in the publishing of studies of such low quality that they’ve seriously compromised the reputation of the journals which published them. I’ll also quote from papers published in other journals which present another side to this debate. And I’ll talk about the nature of proof in general.

Coming from a scientific background, I was as incredulous as anyone else when I first came across this paradoxical therapy. But being open minded and curious, I tried it for myself. That experience was dramatic enough to convince me that here was something worth exploring further. Similar experiences have been common to a vast number of people in homeopathy’s 200-year history. Very few are or were the sort of individuals to turn their backs on respectability and a satisfactory income to pursue a discipline out on the fringes of science (for next to no material gain and more than their fair share of ridicule) unless they felt there was something seriously worth pursuing.

There are many questions still to be answered about homeopathy, not least that it seems to frustrate all attempts to pin it down within the parameters of the biomedical perspective. There are as many still to be asked about what it implies for our understanding of healing in general and for scientific approaches to proof and assessing the validity of experience. What IS certain however, is that the null hypothesis (ie. homeopathy is no more than placebo) is invalid, and that much I can prove to you simply by citing the existing research.

(1) Socks, L M O. Do you ever check the citations themselves? Journal of Time Deficiency 2007;11;255-267