A generalized estimating equation approach to modelling foetal response in developmental toxicity studies when the number of implants is dose dependent

Summary. In developmental toxicity studies with exposure before implantation, the toxin that is used may interfere with the early reproductive process and prevent the implantation of some foetuses that would otherwise have been formed. A manifestation of this effect is a decreasing trend in the number of implants per dam as the dose level increases. Because unformed foetuses are not observable, Dunson proposed a multiple‐imputation approach to estimate the number of missing foetuses. We propose instead to build a model for observable quantities only, namely the observed litter sizes and the observed numbers of deaths or malformations within litters. Using the probabilistic concept of thinning, we express the probability that a foetus fails to implant in terms of the parameters of the litter size distribution. Estimation is by means of generalized estimating equations and takes into account underdispersion of the observed litter sizes and overdispersion of the numbers of foetal deaths or malformations. A combined risk measure that takes into account not just post‐implantation death or malformation but also the possibility of failure to implant is proposed and used to determine the virtually safe dose VSD. It is demonstrated numerically and graphically that ignoring the possibility of unformed foetuses leads to estimates of VSD that are too liberal. The proposed generalized estimating equation approach to estimating VSD and finding lower confidence limits is found to work well in a simulation study. We apply the proposed method to two data sets and compare the results that are obtained with those of existing studies.