IDEAS home Printed from https://ideas.repec.org/a/spr/compst/v28y2013i5p2295-2308.html
   My bibliography  Save this article

Discovering focal regions of slightly-aggregated sparse signals

Author

Listed:
  • Shu-Chun Chen
  • Hsieh Fushing
  • Chii-Ruey Hwang

Abstract

The characteristic aspects of dynamic distortions on a lengthy time series of i.i.d. pure noise when embedded with slightly-aggregating sparse signals are summarized into a significantly shorter recurrence time process of a chosen extreme event. We first employ the Kolmogorov–Smirnov statistic to compare the empirical recurrence time distribution with the null geometry distribution when no signal being present in the original time series. The power of such a hypothesis testing depends on varying degrees of aggregation of sparse signals: from a completely random distribution of singletons to batches of various sizes on the entire temporal span. We demonstrate the Kolmogorov–Smirnov statistic capturing the dynamic distortions due to slightly-aggregating sparse signals better than does Tukey’s Higher Criticism statistic even when the batch size is as small as five. Secondly, after confirming the presence of signals in the pure noise time series, we apply the hierarchical factor segmentation (HFS) algorithm again based on the recurrence time process to compute focal segments that contain a significantly higher intensity of signals than do the rest of the temporal regions. In a computer experiment with a given fixed number of signals, the focal segments identified by the HFS algorithm afford many folds of signal intensity which also critically depend on the degree of aggregation of sparse signals. This ratio information can facilitate better sensitivity, equivalent to a smaller false discovery rate, if the signal-discovering protocol implemented within the computed focal regions is different from that used outside of the focal regions. We also numerically compute the specificity as the total number of signals contained in the computed collection of focal regions, which indicates the inherent difficulty in the task of sparse signal discovery. Copyright Springer-Verlag Berlin Heidelberg 2013

Suggested Citation

  • Shu-Chun Chen & Hsieh Fushing & Chii-Ruey Hwang, 2013. "Discovering focal regions of slightly-aggregated sparse signals," Computational Statistics, Springer, vol. 28(5), pages 2295-2308, October.
  • Handle: RePEc:spr:compst:v:28:y:2013:i:5:p:2295-2308
    DOI: 10.1007/s00180-013-0407-8
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1007/s00180-013-0407-8
    Download Restriction: Access to full text is restricted to subscribers.

    File URL: https://libkey.io/10.1007/s00180-013-0407-8?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Hsieh Fushing & Shu-Chun Chen & Chii-Ruey Hwang, 2012. "Discovering stock dynamics through multidimensional volatility phases," Quantitative Finance, Taylor & Francis Journals, vol. 12(2), pages 213-230, September.
    2. Abramovich, Felix & Benjamini, Yoav, 1996. "Adaptive thresholding of wavelet coefficients," Computational Statistics & Data Analysis, Elsevier, vol. 22(4), pages 351-361, August.
    3. Jin, Jiashun & Cai, T. Tony, 2007. "Estimating the Null and the Proportion of Nonnull Effects in Large-Scale Multiple Comparisons," Journal of the American Statistical Association, American Statistical Association, vol. 102, pages 495-506, June.
    4. Efron, Bradley, 2004. "Large-Scale Simultaneous Hypothesis Testing: The Choice of a Null Hypothesis," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 96-104, January.
    5. Hsieh Fushing & Chen Shu-Chun & Pollard Katherine, 2009. "A Nearly Exhaustive Search for CpG Islands on Whole Chromosomes," The International Journal of Biostatistics, De Gruyter, vol. 5(1), pages 1-24, May.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. T. Tony Cai & Wenguang Sun, 2017. "Optimal screening and discovery of sparse signals with applications to multistage high throughput studies," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(1), pages 197-223, January.
    2. Davide Risso & Liam Purvis & Russell B Fletcher & Diya Das & John Ngai & Sandrine Dudoit & Elizabeth Purdom, 2018. "clusterExperiment and RSEC: A Bioconductor package and framework for clustering of single-cell and other large gene expression datasets," PLOS Computational Biology, Public Library of Science, vol. 14(9), pages 1-16, September.
    3. Lim Johan & Kim Jayoun & Kim Sang-cheol & Yu Donghyeon & Kim Kyunga & Kim Byung Soo, 2012. "Detection of Differentially Expressed Gene Sets in a Partially Paired Microarray Data Set," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(3), pages 1-30, February.
    4. Zhao, Haibing & Fung, Wing Kam, 2016. "A powerful FDR control procedure for multiple hypotheses," Computational Statistics & Data Analysis, Elsevier, vol. 98(C), pages 60-70.
    5. T. Tony Cai & Wenguang Sun & Weinan Wang, 2019. "Covariate‐assisted ranking and screening for large‐scale two‐sample inference," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 81(2), pages 187-234, April.
    6. Chang Yu & Daniel Zelterman, 2020. "Distributions associated with simultaneous multiple hypothesis testing," Journal of Statistical Distributions and Applications, Springer, vol. 7(1), pages 1-17, December.
    7. Ryan Martin, 2021. "A Survey of Nonparametric Mixing Density Estimation via the Predictive Recursion Algorithm," Sankhya B: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 83(1), pages 97-121, May.
    8. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.
    9. Hsieh Fushing & Shu-Chun Chen & Chii-Ruey Hwang, 2014. "Single Stock Dynamics on High-Frequency Data: From a Compressed Coding Perspective," PLOS ONE, Public Library of Science, vol. 9(2), pages 1-12, February.
    10. Pounds Stanley B. & Gao Cuilan L. & Zhang Hui, 2012. "Empirical Bayesian Selection of Hypothesis Testing Procedures for Analysis of Sequence Count Expression Data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(5), pages 1-32, October.
    11. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    12. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    13. Ruth Heller & Saharon Rosset, 2021. "Optimal control of false discovery criteria in the two‐group model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 133-155, February.
    14. Won, Joong-Ho & Lim, Johan & Yu, Donghyeon & Kim, Byung Soo & Kim, Kyunga, 2014. "Monotone false discovery rate," Statistics & Probability Letters, Elsevier, vol. 87(C), pages 86-93.
    15. van Wieringen, Wessel N. & Stam, Koen A. & Peeters, Carel F.W. & van de Wiel, Mark A., 2020. "Updating of the Gaussian graphical model through targeted penalized estimation," Journal of Multivariate Analysis, Elsevier, vol. 178(C).
    16. Ian W. McKeague & Min Qian, 2015. "An Adaptive Resampling Test for Detecting the Presence of Significant Predictors," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 110(512), pages 1422-1433, December.
    17. Chang, Lo-Bin & Geman, Stuart, 2013. "Empirical scaling laws and the aggregation of non-stationary data," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 392(20), pages 5046-5052.
    18. Angela Schörgendorfer & Adam J. Branscum & Timothy E. Hanson, 2013. "A Bayesian Goodness of Fit Test and Semiparametric Generalization of Logistic Regression with Measurement Data," Biometrics, The International Biometric Society, vol. 69(2), pages 508-519, June.
    19. Travis J. Berge & Shu-Chun Chen & Hsieh Fushing & Òscar Jordà, 2010. "A chronology of international business cycles through non-parametric decoding," Research Working Paper RWP 11-13, Federal Reserve Bank of Kansas City.
    20. Han, Bing & Dalal, Siddhartha R., 2012. "A Bernstein-type estimator for decreasing density with application to p-value adjustments," Computational Statistics & Data Analysis, Elsevier, vol. 56(2), pages 427-437.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:compst:v:28:y:2013:i:5:p:2295-2308. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.