Bayesian analysis of tests with unknown specificity and sensitivity

When testing for a rare disease, prevalence estimates can be highly sensitive to uncertainty in the specificity and sensitivity of the test. Bayesian inference is a natural way to propagate these uncertainties, with hierarchical modelling capturing variation in these parameters across experiments. Another concern is the people in the sample not being representative of the general population. Statistical adjustment cannot without strong assumptions correct for selection bias in an opt‐in sample, but multilevel regression and post‐stratification can at least adjust for known differences between the sample and the population. We demonstrate hierarchical regression and post‐stratification models with code in Stan and discuss their application to a controversial recent study of SARS‐CoV‐2 antibodies in a sample of people from the Stanford University area. Wide posterior intervals make it impossible to evaluate the quantitative claims of that study regarding the number of unreported infections. For future studies, the methods described here should facilitate more accurate estimates of disease prevalence from imperfect tests performed on non‐representative samples.