IDEAS home Printed from https://ideas.repec.org/a/bla/jorssb/v84y2022i5p1608-1639.html
   My bibliography  Save this article

Calibrating the scan statistic: Finite sample performance versus asymptotics

Author

Listed:
  • Guenther Walther
  • Andrew Perry

Abstract

We consider the problem of detecting an elevated mean on an interval with unknown location and length in the univariate Gaussian sequence model. Recent results have shown that using scale‐dependent critical values for the scan statistic allows to attain asymptotically optimal detection simultaneously for all signal lengths, thereby improving on the traditional scan, but this procedure has been criticised for losing too much power for short signals. We explain this discrepancy by showing that these asymptotic optimality results will necessarily be too imprecise to discern the performance of scan statistics in a practically relevant way, even in a large sample context. Instead, we propose to assess the performance with a new finite sample criterion. We then present three calibrations for scan statistics that perform well across a range of relevant signal lengths: The first calibration uses a particular adjustment to the critical values and is therefore tailored to the Gaussian case. The second calibration uses a scale‐dependent adjustment to the significance levels rather than to the critical values and this adjustment is therefore applicable to arbitrary known null distributions. The third calibration restricts the scan to a particular sparse subset of the scan windows and then applies a weighted Bonferroni adjustment to the corresponding test statistics. This Bonferroni scan is also applicable to arbitrary null distributions and in addition is very simple to implement. We show how to apply these calibrations for scanning in a number of distributional settings: for normal observations with an unknown baseline and a known or unknown constant variance, for observations from a natural exponential family, for potentially heteroscadastic observations from a symmetric density by employing self‐normalisation in a novel way, and for exchangeable observations using tests based on permutations, ranks or signs.

Suggested Citation

  • Guenther Walther & Andrew Perry, 2022. "Calibrating the scan statistic: Finite sample performance versus asymptotics," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(5), pages 1608-1639, November.
  • Handle: RePEc:bla:jorssb:v:84:y:2022:i:5:p:1608-1639
    DOI: 10.1111/rssb.12549
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/rssb.12549
    Download Restriction: no

    File URL: https://libkey.io/10.1111/rssb.12549?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Klaus Frick & Axel Munk & Hannes Sieling, 2014. "Multiscale change point inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 76(3), pages 495-580, June.
    2. Kabluchko, Zakhar, 2011. "Extremes of the standardized Gaussian noise," Stochastic Processes and their Applications, Elsevier, vol. 121(3), pages 515-533, March.
    3. Camilo Rivera & Guenther Walther, 2013. "Optimal detection of a jump in the intensity of a Poisson process or in a density with likelihood ratio statistics," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 40(4), pages 752-769, December.
    4. Ery Arias-Castro & Rui M. Castro & Ervin Tánczos & Meng Wang, 2018. "Distribution-Free Detection of Structured Anomalies: Permutation and Rank-Based Scans," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 113(522), pages 789-801, April.
    5. Joseph I. Naus & Sylvan Wallenstein, 2004. "Multiple Window and Cluster Size Scan Procedures," Methodology and Computing in Applied Probability, Springer, vol. 6(4), pages 389-400, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Florian Pein & Hannes Sieling & Axel Munk, 2017. "Heterogeneous change point inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1207-1227, September.
    2. Cho, Haeran & Kirch, Claudia, 2022. "Bootstrap confidence intervals for multiple change points based on moving sum procedures," Computational Statistics & Data Analysis, Elsevier, vol. 175(C).
    3. S Kovács & P Bühlmann & H Li & A Munk, 2023. "Seeded binary segmentation: a general methodology for fast and optimal changepoint detection," Biometrika, Biometrika Trust, vol. 110(1), pages 249-256.
    4. Wu, Tung-Lung & Glaz, Joseph, 2015. "A new adaptive procedure for multiple window scan statistics," Computational Statistics & Data Analysis, Elsevier, vol. 82(C), pages 164-172.
    5. Markos V. Koutras & Demetrios P. Lyberopoulos, 2018. "Asymptotic results for jump probabilities associated to the multiple scan statistic," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 70(5), pages 951-968, October.
    6. Bill Russell & Dooruj Rambaccussing, 2019. "Breaks and the statistical process of inflation: the case of estimating the ‘modern’ long-run Phillips curve," Empirical Economics, Springer, vol. 56(5), pages 1455-1475, May.
    7. Salvatore Fasola & Vito M. R. Muggeo & Helmut Küchenhoff, 2018. "A heuristic, iterative algorithm for change-point detection in abrupt change models," Computational Statistics, Springer, vol. 33(2), pages 997-1015, June.
    8. Michael Messer, 2022. "Bivariate change point detection: Joint detection of changes in expectation and variance," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(2), pages 886-916, June.
    9. Wu Wang & Xuming He & Zhongyi Zhu, 2020. "Statistical inference for multiple change‐point models," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1149-1170, December.
    10. Sokbae Lee & Myung Hwan Seo & Youngki Shin, 2016. "The lasso for high dimensional regression with a possible change point," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 78(1), pages 193-210, January.
    11. Kang-Ping Lu & Shao-Tung Chang, 2021. "Robust Algorithms for Change-Point Regressions Using the t -Distribution," Mathematics, MDPI, vol. 9(19), pages 1-28, September.
    12. Andreas Anastasiou & Piotr Fryzlewicz, 2022. "Detecting multiple generalized change-points by isolating single ones," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 85(2), pages 141-174, February.
    13. Yudong Chen & Tengyao Wang & Richard J. Samworth, 2022. "High‐dimensional, multiscale online changepoint detection," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 84(1), pages 234-266, February.
    14. Stefan Albert & Michael Messer & Julia Schiemann & Jochen Roeper & Gaby Schneider, 2017. "Multi-Scale Detection of Variance Changes in Renewal Processes in the Presence of Rate Change Points," Journal of Time Series Analysis, Wiley Blackwell, vol. 38(6), pages 1028-1052, November.
    15. Chen, Yudong & Wang, Tengyao & Samworth, Richard J., 2022. "High-dimensional, multiscale online changepoint detection," LSE Research Online Documents on Economics 113665, London School of Economics and Political Science, LSE Library.
    16. Lu Shaochuan, 2023. "Scalable Bayesian Multiple Changepoint Detection via Auxiliary Uniformisation," International Statistical Review, International Statistical Institute, vol. 91(1), pages 88-113, April.
    17. Camillo Cammarota, 2017. "Estimating the turning point location in shifted exponential model of time series," Journal of Applied Statistics, Taylor & Francis Journals, vol. 44(7), pages 1269-1281, May.
    18. Claudia Kirch & Christina Stoehr, 2022. "Sequential change point tests based on U‐statistics," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 49(3), pages 1184-1214, September.
    19. Kang-Ping Lu & Shao-Tung Chang, 2023. "An Advanced Segmentation Approach to Piecewise Regression Models," Mathematics, MDPI, vol. 11(24), pages 1-23, December.
    20. Yi-Shen Lin & Xenos Chang-Shuo Lin & Daniel Wei-Chung Miao & Yi-Ching Yao, 2020. "Corrected Discrete Approximations for Multiple Window Scan Statistics of One-Dimensional Poisson Processes," Methodology and Computing in Applied Probability, Springer, vol. 22(1), pages 237-265, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:jorssb:v:84:y:2022:i:5:p:1608-1639. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://edirc.repec.org/data/rssssea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.