Prosecutor’s Fallacies and Relative Probabilities

In the United Kingdom some years ago a mother of two deceased children was convicted, in part, because evidence was given that the probability of her two children dying through natural means or cot death (her defence) was very low – as low as 1 in 73 million.

The woman was Sally Clark, a solicitor at the time.   The disturbing decision went on appeal, but she never recovered from the trauma and died after release from prison.

Well-meaning mathematicians have offered three broad opinions on the relevance of the statistical data to the case:

  1.  The 1 in 73 million probability was over-stated; it was based on wrong assumptions about independence of each of the two children’s death; that the true rate of cot death was closer to 1 in 100,000.
  2. The claim that a person is guilty of the charges because the nature of the defence has such a low incidence rate, generally, is an example of the ‘prosecutor’s fallacy’; and
  3. To avoid the prosecutor’s fallacy in such a case, we should look at how relatively likely the prosecution case is compared to the defence’s case.

See Plus Magazine

The third point relates to proof in legal proceedings and is not something entirely within the domain of mathematics.  

The authors of the Plus Magazine offered the explanation that Bayes’ Rule would allow the calculation of the relative likelihoods of the kinds of events forming the basis for the prosecution and defence cases.  These relative likelihoods, based on a comparison of how frequently different kinds of events occur in the general population.

I don’t accept that these relative probabilities are useful metrics, at least not in such a crude form.   The main objection is that in Court cases, we are not simply trying to guess at whether we are randomly pulling a ‘criminal’ or an ‘innocent’ person from a general population.   We are not attempting a classification exercise, by comparing which of two unknown events would be more likely, before we know the outcome.  There’s nothing wrong with the advice to look at how likely it is that other events might have occurred, viewed at the same time, and looking forward into the future.  However, I think that such an approach leads us to look at the Court’s role in a slightly inaccurate way.

The problem with the third part of the article, namely the use of relative probabilities, to answer the prosecutor’s fallacy, is that it continues to ask the wrong questions, and they are:

  1. What is the probability that something might happen, in the future, if we don’t have any further information?  
  2. What is the probability that some other event might happen, in the future, if we don’t have any further information?
  3. Which of these probabilities is the highest?

This merely replaces one line of fallacious reasoning with another.  In a Court case, we do not want to predict the prospect that a randomly chosen person, from the general population, is more likely to have been a criminal than an innocent person.  What if the general population statistics had been different?  That would not have changed the innocence of Ms Clark.   From her perspective, the only historical event that had occurred involved her innocence, with a probability of 100%.

The jury’s role is not to make gambling wagers on innocence based on sampled statistics that have nothing to do with the trajectory of historical events in a specific case.  The jury is to weigh up their own subjective assessment of whether a historical event occurred, based on the evidence that relates to the specific evidence and the specific accused.  General statistics for the same kind of offence should never be used as the basis for a conviction, at least not without asking how that helps avoid convicting an innocent person.

A trial is not trying to estimate the occurrence of a random event, but trying to unravel and resolve uncertainties about what occurred in a particular setting, at a particular time, with specific people.

A Court case is more analogous to a situation in which we are attempting to confirm a hypothesis that an event (and the events leading up to it, implicitly), have already occurred.    We are not trying to predict whether a series of events will occur, before they happen.

From time to time, people trying to prove historical facts do suddenly become excited by the notion that anything that happens in the world, if you go back to the time when you did not know anything, appear extremely unlikely.   The more we try to predict the future, in apparent ignorance of all other relevant information, the more unpredictable it seems.   The sobering insight is that so too are all the other possible events.

Leave a Reply

Your email address will not be published. Required fields are marked *