IDEAS home Printed from https://ideas.repec.org/a/spr/metcap/v18y2016i3d10.1007_s11009-015-9447-6.html
   My bibliography  Save this article

Scan Statistic Tail Probability Assessment Based on Process Covariance and Window Size

Author

Listed:
  • Anat Reiner-Benaim

    (University of Haifa)

Abstract

A scan statistic is examined for the purpose of testing the existence of a global peak in a random process with dependent variables of any distribution. The scan statistic tail probability is obtained based on the covariance of the moving sums process, thereby accounting for the spatial nature of the data as well as the size of the searching window. Exact formulas linking this covariance to the window size and the correlation coefficient are developed under general, common and auto covariance structures of the variables in the original process. The implementation and applicability of the formulas are demonstrated on multiple processes of t-statistics, treating also the case of unknown covariance. A sensitivity analysis provides further insight into the variant interaction of the tail probability with the influence parameters. An R code for the tail probability computation and the data analysis is offered within the supplementary material.

Suggested Citation

  • Anat Reiner-Benaim, 2016. "Scan Statistic Tail Probability Assessment Based on Process Covariance and Window Size," Methodology and Computing in Applied Probability, Springer, vol. 18(3), pages 717-745, September.
    • Handle: RePEc:spr:metcap:v:18:y:2016:i:3:d:10.1007_s11009-015-9447-6
    DOI: 10.1007/s11009-015-9447-6
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11009-015-9447-6
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11009-015-9447-6?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Daniel Yekutieli & Anat Reiner‐Benaim & Yoav Benjamini & Gregory I. Elmer & Neri Kafkafi & Noah E. Letwin & Norman H. Lee, 2006. "Approaches to multiplicity issues in complex research in microarray analysis," Statistica Neerlandica, Netherlands Society for Statistics and Operations Research, vol. 60(4), pages 414-437, November.
    2. Chan, Hock Peng & Zhang, Nancy Ruonan, 2007. "Scan Statistics With Weighted Observations," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 595-602, June.
    3. Christopher R. Genovese & Kathryn Roeder & Larry Wasserman, 2006. "False discovery control with p-value weighting," Biometrika, Biometrika Trust, vol. 93(3), pages 509-524, September.
    4. Haiman, George & Preda, Cristian, 2013. "One dimensional scan statistics generated by some dependent stationary sequences," Statistics & Probability Letters, Elsevier, vol. 83(5), pages 1457-1463.
    5. Huang, Lan & Tiwari, Ram C. & Zou, Zhaohui & Kulldorff, Martin & Feuer, Eric J., 2009. "Weighted Normal Spatial Scan Statistic for Heterogeneous Population Data," Journal of the American Statistical Association, American Statistical Association, vol. 104(487), pages 886-898.
    6. Taylor, Jonathan E. & Worsley, Keith J., 2007. "Detecting Sparse Signals in Random Fields, With an Application to Brain Mapping," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 913-928, September.
    7. Ledoit, Olivier & Wolf, Michael, 2003. "Improved estimation of the covariance matrix of stock returns with an application to portfolio selection," Journal of Empirical Finance, Elsevier, vol. 10(5), pages 603-621, December.
    8. D. O. Siegmund & N. R. Zhang & B. Yakir, 2011. "False discovery rate for scanning statistics," Biometrika, Biometrika Trust, vol. 98(4), pages 979-985.
    9. Schäfer Juliane & Strimmer Korbinian, 2005. "A Shrinkage Approach to Large-Scale Covariance Matrix Estimation and Implications for Functional Genomics," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 4(1), pages 1-32, November.
    10. M. Perone Pacifico & C. Genovese & I. Verdinelli & L. Wasserman, 2004. "False Discovery Control for Random Fields," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 1002-1014, December.
    11. Joseph Glaz & Joseph Naus & Xiao Wang, 2012. "Approximations and Inequalities for Moving Sums," Methodology and Computing in Applied Probability, Springer, vol. 14(3), pages 597-616, September.
    12. Hao Chen & Haipeng Xing & Nancy R Zhang, 2011. "Estimation of Parent Specific DNA Copy Number in Tumors using High-Density Genotyping Arrays," PLOS Computational Biology, Public Library of Science, vol. 7(1), pages 1-15, January.
    13. M. Koutras & V. Alexandrou, 1995. "Runs, scans and URN model distributions: A unified Markov chain approach," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 47(4), pages 743-766, December.
    14. Efron, Bradley, 2010. "Correlated z-Values and the Accuracy of Large-Scale Statistical Estimates," Journal of the American Statistical Association, American Statistical Association, vol. 105(491), pages 1042-1055.
    15. Sunduz Keles & Mark van der Laan & Sandrine Dudoit & Simon Cawley, 2004. "Multiple Testing Methods For ChIP-Chip High Density Oligonucleotide Array Data," U.C. Berkeley Division of Biostatistics Working Paper Series 1147, Berkeley Electronic Press.
    16. Efron, Bradley, 2007. "Correlation and Large-Scale Simultaneous Significance Testing," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 93-103, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    2. Jianqing Fan & Yuan Liao & Martina Mincheva, 2013. "Large covariance estimation by thresholding principal orthogonal complements," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 75(4), pages 603-680, September.
    3. Hannart, Alexis & Naveau, Philippe, 2014. "Estimating high dimensional covariance matrices: A new look at the Gaussian conjugate framework," Journal of Multivariate Analysis, Elsevier, vol. 131(C), pages 149-162.
    4. Wang, Christina Dan & Chen, Zhao & Lian, Yimin & Chen, Min, 2022. "Asset selection based on high frequency Sharpe ratio," Journal of Econometrics, Elsevier, vol. 227(1), pages 168-188.
    5. Pan-Jun Kim & Nathan D Price, 2011. "Genetic Co-Occurrence Network across Sequenced Microbes," PLOS Computational Biology, Public Library of Science, vol. 7(12), pages 1-9, December.
    6. Ledoit, Olivier & Wolf, Michael, 2017. "Numerical implementation of the QuEST function," Computational Statistics & Data Analysis, Elsevier, vol. 115(C), pages 199-223.
    7. Bartosz Kaszuba, 2012. "Empirical Comparison of Robust Portfolios’ Investment Effects," The Review of Finance and Banking, Academia de Studii Economice din Bucuresti, Romania / Facultatea de Finante, Asigurari, Banci si Burse de Valori / Catedra de Finante, vol. 5(1), pages 047-061, June.
    8. Bo Zhao & Joseph Glaz, 2016. "Scan Statistics for Detecting a Local Change in Variance for Normal Data with Known Variance," Methodology and Computing in Applied Probability, Springer, vol. 18(2), pages 563-573, June.
    9. Yi-Shen Lin & Xenos Chang-Shuo Lin & Daniel Wei-Chung Miao & Yi-Ching Yao, 2020. "Corrected Discrete Approximations for Multiple Window Scan Statistics of One-Dimensional Poisson Processes," Methodology and Computing in Applied Probability, Springer, vol. 22(1), pages 237-265, March.
    10. Bergsteinsson, Hjörleifur G. & Møller, Jan Kloppenborg & Nystrup, Peter & Pálsson, Ólafur Pétur & Guericke, Daniela & Madsen, Henrik, 2021. "Heat load forecasting using adaptive temporal hierarchies," Applied Energy, Elsevier, vol. 292(C).
    11. Wang, Xiao & Zhao, Bo & Glaz, Joseph, 2014. "A multiple window scan statistic for time series models," Statistics & Probability Letters, Elsevier, vol. 94(C), pages 196-203.
    12. Lim Johan & Kim Jayoun & Kim Sang-cheol & Yu Donghyeon & Kim Kyunga & Kim Byung Soo, 2012. "Detection of Differentially Expressed Gene Sets in a Partially Paired Microarray Data Set," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(3), pages 1-30, February.
    13. Bailey, Natalia & Pesaran, M. Hashem & Smith, L. Vanessa, 2019. "A multiple testing approach to the regularisation of large sample correlation matrices," Journal of Econometrics, Elsevier, vol. 208(2), pages 507-534.
    14. Ikeda, Yuki & Kubokawa, Tatsuya & Srivastava, Muni S., 2016. "Comparison of linear shrinkage estimators of a large covariance matrix in normal and non-normal distributions," Computational Statistics & Data Analysis, Elsevier, vol. 95(C), pages 95-108.
    15. Sairam Rayaprolu & Zhiyi Chi, 2021. "False Discovery Variance Reduction in Large Scale Simultaneous Hypothesis Tests," Methodology and Computing in Applied Probability, Springer, vol. 23(3), pages 711-733, September.
    16. Dimitri Yatsenko & Krešimir Josić & Alexander S Ecker & Emmanouil Froudarakis & R James Cotton & Andreas S Tolias, 2015. "Improved Estimation and Interpretation of Correlations in Neural Circuits," PLOS Computational Biology, Public Library of Science, vol. 11(3), pages 1-28, March.
    17. Wenguang Sun & T. Tony Cai, 2009. "Large‐scale multiple testing under dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 71(2), pages 393-424, April.
    18. Carel F. W. Peeters & Mark A. Wiel & Wessel N. Wieringen, 2020. "The spectral condition number plot for regularization parameter evaluation," Computational Statistics, Springer, vol. 35(2), pages 629-646, June.
    19. Reiner-Benaim Anat & Davis Ronald W. & Juneau Kara, 2014. "Scan statistics analysis for detection of introns in time-course tiling array data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 13(2), pages 173-190, April.
    20. Matthias M. M. Buehlmaier & Kit Pong Wong, 2020. "Should investors join the index revolution? Evidence from around the world," Journal of Asset Management, Palgrave Macmillan, vol. 21(3), pages 192-218, May.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:metcap:v:18:y:2016:i:3:d:10.1007_s11009-015-9447-6. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.