Stats in law has been one of my bug-bears for quite some time. Lawyers, being very language based [as a rule] don’t seem to have any mathematical skills. My own law class, struggle to work out a basic % calculation.
Statistical significance, particularly when looking at ‘causation’ in law, has been a killing field for accurate legal argument.
As the American Statistical Association said,
Researchers often wish to turn a p-value into a statement about the truth of a null hypothesis, or about the probability that random chance produced the observed data. The p-value is neither. It is a statement about data in relation to a specified hypothetical explanation, and is not a statement about the explanation itself. …
Smaller p-values do not necessarily imply the presence of larger or more important effects, and larger p-values do not imply a lack of importance or even lack of effect. Any effect, no matter how tiny, can produce a small p-value if the sample size or measurement precision is high enough, and large effects may produce unimpressive p-values if the sample size is small or measurements are imprecise.
The Court in the In re Chantix litigation got it exactly right:
While the defendant repeatedly harps on the importance of statistically significant data, the United States Supreme Court recently stated that “[a] lack of statistically significant data does not mean that medical experts have no reliable basis for inferring a causal link between a drug and adverse events …. medical experts rely on other evidence to establish an inference of causation.” Matrixx Initiatives, Inc. v. Siracusano,––– U.S. ––––, 131 S.Ct. 1309, 1319, 179 L.Ed.2d 398 (2011). The Court further recognised that courts “frequently permit expert testimony on causation based on evidence other than statistical significance.” Id.; citing Wells v. Ortho Pharmaceutical Corp., 788 F.2d 741, 744–745 (11th Cir.1986). Hence, the court does not find the defendant’s argument that Dr. Furberg “cannot establish a valid statistical association between Chantix and serious neuropsychiatric events” to be a persuasive reason to exclude his opinion, even if the court found the same to be true.