While musing on the nature of evidence, and what is or is not considered ‘evidence’ in all the various contexts evidence is used, I came across this excellent rapid response in the BMJ back in 2004 by Clifford Miller, “a practising English lawyer, graduate in physics and a sometime examining lecturer on law, standards and ethics (particularly, the law of evidence) to Masters student technologists at the Imperial College of Science Technology and Medicine”.

He writes, in the context of the imputed association between MMR vaccination and the onset of autistic states, on “The Unreliability of Scientific Papers as Evidence”:

“Reliable evidence is that which is authentic, accurate and complete. In short, scientific evidence is incomplete if used for purposes outside the strict confines of science because it fails to take account of evidence of lay witnesses of the facts and is hence only applicable to the narrow and specific confines of scientific enquiry and not the broader ones found in other fields of human endeavour.

“Examples in point include the parental evidence of symptoms in the MMR cases or that of Gulf War veterans about their symptoms. A court (or the Legal Services Commission in the case of MMR) in contrast, ought to take that oral evidence into account for the very reasons science dismisses it. The point, unfortunately is not as well taken by our legal system as it might be. We have seen this recently with the Legal Services Commission in the MMR cases and in the cases of Gulf War veterans.

“Science treats evidence of lay witnesses of fact as inadmissible (as ‘anecdotal’ only) for reasons which are inapplicable in Court, but science does so for two main reasons. The higher scientific standard of proof (in effect, irrefutability) only admits evidence which can be tested scientifically for reliability. Oral witness evidence is discounted by medical science because medical scientific method does not currently have or recognise a mechanism for testing oral evidence to the scientific standard and so, for the sake of rigour, excludes it.

“Neither of these propositions apply in Court. Evidence of the direct witness of the fact, whether oral, or more frequently now, by way of written statement, is always admissible and is, in fact, the keystone of the trial system of evidence and the primary source of information a court uses to make decisions of fact. The Court has and applies its own mechanisms for testing witness evidence (eg. cross-examination). Further, the Court applies a far lower standard of proof, namely a balance of probability and not the unnecessarily high one of irrefutability applied by science.

[…]

“Governments also take advantage of the confusion and often use the term ‘evidence’ interchangeably with ‘proof’ when dismissing evidence they choose not to agree with or set unreasonably high standards of proof for the kind of decision required. The press and public alike are continually hoodwinked by this approach.

“In law ‘evidence’ is nothing more than information. It is information which one party proposes in support of, or to undermine, a disputed proposition. ‘Proof’, however, depends upon the decision-making process concerned. For the public interest, the standard of proof is sometimes based on risk and sometimes on other factors. In civil courts it is ‘balance of probability’. In criminal it is ‘beyond reasonable doubt’. And science requires irrefutable proof: a remarkably high standard.

It is a fundamental error to apply the wrong standard of proof to the decision making process concerned and yet it seems to happen regularly.

“For issues of public safety, such as medicines like MMR or vaccines in the Gulf War, or the BSE crisis, the risk standard ought to be applied.

“However, instead, we, the public, are told frequently by officials in government there is no scientific evidence of a causal link between one thing and another. Whereas, often evidence to the contrary does in fact exist, it is not evidence that the officialdom concerned may either choose to or sometimes be at liberty to accept as proof of the issue. This is much the same for the BSE crisis with the government as it was for the Courts in relation to Gulf War syndrome or the Legal Services Commission for MMR. Whereas in the case of courts, the court has to rely on the expert evidence presented, in the case of public health officials like the Chief Medical Officer, he is in a position to assess the reliability himself, with the aid of his own experts if necessary. However, in the latter case, the risk standard of proof ought to be applied to decision making in the public interest rather than the scientific standard, which is only applicable to proof in science.

“In the scientific context, the only answer to a scientific issue that scientific journals should involve themselves in is a scientific one and they should only trouble themselves with the scientific standard of proof. If MMR did not cause autism, or vaccines in the Gulf War did not cause other problems, then it is for scientific journals to publish irrefutable scientific proof of what ails the 1000 or so children and the numerous afflicted Gulf War veterans.

“The current political debates about these kinds of issues are ones science could answer, if only the scientists got on with it and stopped playing politics.

“It is, for example, no answer to Wakefield to claim there is no scientific evidence of a link between MMR and autism. That just shows science has not found one that it can accept as proven to its very high standard of proof. It does not prove there are none, nor that there is no proof to other more realistic and practical standards for day-to-day decision making. It also leaves the public confused and distrustful of science.

“Buried in the MMR debate and little known to the general public is formal confirmation of a link between immunisation and the so-called allergy epidemics in the developed world. According to the US National Academies’ Institute of Medicine (IoM) Immunization Safety Review Committee (1), for at least two years it has been known that current vaccination programmes can expose children to risk of various problems ranging from allergy to infection. The IoM have also confirmed (2) that reasonable theories exist to explain how too many immunizations can overwhelm an infant’s immune system.

“A clear indication of the possibility of the existence of a causal connection between vaccination and the emergence of the various allergy and other issues over the last 20 years is the contemporaneous substantial increase in vaccinations as reported by the IoM (3). This shows an increase from 4 vaccinations per child in 1980 to up to 20 now.

“Whilst the IoM considered (4), as regards asthma in particular, and allergies in general, it had inadequate evidence to accept or reject a causal relationship, it accepted there is cause to consider that there might be a connection. Effectively, all the IoM statement amounts to is an admission by the most authoritative governmental authority in the US that they will not accept any evidence unless it provides the answer to a scientific standard of proof, and until someone produces that proof, they will not apply a risk standard, such that it is immunization as usual for children.

“The IoM’s conclusion is also not a reliable one for government to apply to the risk standard of proof because the IoM rely upon the scientific standard of proof and that is the wrong standard to apply for a decision based on risk. Irrefutability is too high a hurdle. Similarly, parents taking practical day-to-day decisions risk their child’s health if they wait for scientific proof, because proof to such a standard also takes too long to be produced.

Quite. Yet note that the MMR vaccination was foisted upon the population at large on the basis of those same scientific standards of evidence. So scientific standards of evidence conclude the vaccine has “proven” efficacy and “safety”, despite the fact that trials of the vaccine are limited and have no long-term follow-up. Scientific standards of evidence also conclude that there’s no adequate association between MMR vaccination and autism, despite the fact that documentary evidence acceptable as prime evidence in a court of law presents compelling evidence to the contrary.

What’s wrong with this picture? The aim of science is to arrive at an explanation and understanding of observed phenomena and their causal mechanisms that is, above all, utterly faithful to those observed phenomena. Not politics, not commercial expediency, not pre-existing theories or biases. The observed phenomena. Full stop, end of story. Consequently any hypothesis that meets those conditions should be acceptable to anyone observing the phenomena under consideration because the degree of fit will be such as to render any alternative explanation superfluous and irrelevant.

But when there is such substantial variance between the observed phenomena according to day-to-day standards of evidence and the preferred ‘scientific’ hypothesis, then something has to have gone badly wrong, and the scientific process has become subverted by something other than the aim to explain the observed phenomena. If the two are not congruent, then the scientific proces has failed.