Bayesian inference for stochastic differential equation mixed effects models of a tumour xenography study

We consider Bayesian inference for stochastic differential equation mixed effects models (SDEMEMs) exemplifying tumour response to treatment and regrowth in mice. We produce an extensive study on how an SDEMEM can be fitted by using both exact inference based on pseudo‐marginal Markov chain Monte Carlo sampling and approximate inference via Bayesian synthetic likelihood (BSL). We investigate a two‐compartments SDEMEM, corresponding to the fractions of tumour cells killed by and survived on a treatment. Case‐study data consider a tumour xenography study with two treatment groups and one control, each containing 5–8 mice. Results from the case‐study and from simulations indicate that the SDEMEM can reproduce the observed growth patterns and that BSL is a robust tool for inference in SDEMEMs. Finally, we compare the fit of the SDEMEM with a similar ordinary differential equation model. Because of small sample sizes, strong prior information is needed to identify all model parameters in the SDEMEM and it cannot be determined which of the two models is the better in terms of predicting tumour growth curves. In a simulation study we find that with a sample of 17 mice per group BSL can identify all model parameters and distinguish treatment groups.