IDEAS home Printed from https://ideas.repec.org/a/oup/biomet/v97y2010i2p419-433.html
   My bibliography  Save this article

Efficient scalable schemes for monitoring a large number of data streams

Author

Listed:
  • Y. Mei

Abstract

The sequential changepoint detection problem is studied in the context of global online monitoring of a large number of independent data streams. We are interested in detecting an occurring event as soon as possible, but we do not know when the event will occur, nor do we know which subset of data streams will be affected by the event. A family of scalable schemes is proposed based on the sum of the local cumulative sum, cusum , statistics from each individual data stream, and is shown to asymptotically minimize the detection delays for each and every possible combination of affected data streams, subject to the global false alarm constraint. The usefulness and limitations of our asymptotic optimality results are illustrated by numerical simulations and heuristic arguments. The Appendices contain a probabilistic result on the first epoch to simultaneous record values for multiple independent random walks. Copyright 2010, Oxford University Press.

Suggested Citation

  • Y. Mei, 2010. "Efficient scalable schemes for monitoring a large number of data streams," Biometrika, Biometrika Trust, vol. 97(2), pages 419-433.
    • Handle: RePEc:oup:biomet:v:97:y:2010:i:2:p:419-433
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/biomet/asq010
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Jay Bartroff & Jinlin Song, 2016. "A Rejection Principle for Sequential Tests of Multiple Hypotheses Controlling Familywise Error Rates," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(1), pages 3-19, March.
    2. Alexander G. Tartakovsky, 2019. "Asymptotically Optimal Quickest Change Detection in Multistream Data—Part 1: General Stochastic Models," Methodology and Computing in Applied Probability, Springer, vol. 21(4), pages 1303-1336, December.
    3. Yudong Chen & Tengyao Wang & Richard J. Samworth, 2022. "High‐dimensional, multiscale online changepoint detection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(1), pages 234-266, February.
    4. Chen, Yunxiao & Lee, Yi-Hsuan & Li, Xiaoou, 2022. "Item pool quality control in educational testing: change point model, compound risk, and sequential detection," LSE Research Online Documents on Economics 112498, London School of Economics and Political Science, LSE Library.
    5. Bertille Follain & Tengyao Wang & Richard J. Samworth, 2022. "High‐dimensional changepoint estimation with heterogeneous missingness," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(3), pages 1023-1055, July.
    6. Du, Lilun & Wen, Mengtao, 2023. "False discovery rate approach to dynamic change detection," Journal of Multivariate Analysis, Elsevier, vol. 198(C).
    7. Follain, Bertille & Wang, Tengyao & Samworth, Richard J., 2022. "High-dimensional changepoint estimation with heterogeneous missingness," LSE Research Online Documents on Economics 115014, London School of Economics and Political Science, LSE Library.
    8. Chen, Yudong & Wang, Tengyao & Samworth, Richard J., 2022. "High-dimensional, multiscale online changepoint detection," LSE Research Online Documents on Economics 113665, London School of Economics and Political Science, LSE Library.
    9. Yunxiao Chen & Yi-Hsuan Lee & Xiaoou Li, 2022. "Item Pool Quality Control in Educational Testing: Change Point Model, Compound Risk, and Sequential Detection," Journal of Educational and Behavioral Statistics, , vol. 47(3), pages 322-352, June.
    10. Hahn, Georg, 2022. "Online multivariate changepoint detection with type I error control and constant time/memory updates per series," Statistics & Probability Letters, Elsevier, vol. 181(C).
    11. Cui, Junfeng & Wang, Guanghui & Zou, Changliang & Wang, Zhaojun, 2023. "Change-point testing for parallel data sets with FDR control," Computational Statistics & Data Analysis, Elsevier, vol. 182(C).
    12. Lu, Zexian & Chen, Yunxiao & Li, Xiaoou, 2022. "Optimal parallel sequential change detection under generalized performance measures," LSE Research Online Documents on Economics 118348, London School of Economics and Political Science, LSE Library.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:biomet:v:97:y:2010:i:2:p:419-433. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/biomet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.