Signal Detection Based on Time to Onset Algorithm in Spontaneous Reporting System of China

Introduction The method of time-to-onset (TTO) has been proposed to overcome the drawbacks of traditional disproportionality analyses (DPAs), and it has been used for detecting safety signals of vaccines and some non-vaccine products in spontaneous reporting systems (SRSs). However, there is no consensus on its superiority over DPAs. Further, it is still not clear whether this novel approach can be generalized to the entire national SRS database. Objective The purpose of this study was to generalize the TTO method to the Chinese SRS and to identify suitable parameters for its optimal performance. Methods Reports submitted to the national SRS of China in 2014 were used as the data source for analysis. We evaluated the performance of TTO by using product labels as proxies for the gold standard. A series of values of significance level and time windows were explored to identify the most suitable parameters for TTO based on Youden’s index, a statistic that summarizes the performance of a diagnostic test. Additionally, we compared TTO with traditional DPAs and explored the characteristics of signals detected by these methods. Results Compared with DPAs, TTO had a lower sensitivity, but higher specificity and positive predictive value. At a significance level of 0.2 and no restrictions on time windows, TTO had the highest Youden’s index. The kappa coefficients between TTO and DPAs were rather low, indicating poor agreement between the two methods. More than 30% of the true signals detected by TTO were not identified by DPAs. Furthermore, TTO needed more number of reports to be able to detect signals. Conclusions TTO can detect signals missed by traditional DPAs and could be an important complementary tool to the currently used DPAs in the SRS of China. We recommend a significance level of 0.2 and no restrictions on time windows for TTO.