IDEAS home Printed from https://ideas.repec.org/a/spr/alstar/v102y2018i2d10.1007_s10182-017-0297-0.html
   My bibliography  Save this article

Closure properties of classes of multiple testing procedures

Author

Listed:
  • Georg Hahn

    (Imperial College London)

Abstract

Statistical discoveries are often obtained through multiple hypothesis testing. A variety of procedures exists to evaluate multiple hypotheses, for instance the ones of Benjamini–Hochberg, Bonferroni, Holm or Sidak. We are particularly interested in multiple testing procedures with two desired properties: (solely) monotonic and well-behaved procedures. This article investigates to which extent the classes of (monotonic or well-behaved) multiple testing procedures, in particular the subclasses of so-called step-up and step-down procedures, are closed under basic set operations, specifically the union, intersection, difference and the complement of sets of rejected or non-rejected hypotheses. The present article proves two main results: First, taking the union or intersection of arbitrary (monotonic or well-behaved) multiple testing procedures results in new procedures which are monotonic but not well-behaved, whereas the complement or difference generally preserves neither property. Second, the two classes of (solely monotonic or well-behaved) step-up and step-down procedures are closed under taking the union or intersection, but not the complement or difference.

Suggested Citation

  • Georg Hahn, 2018. "Closure properties of classes of multiple testing procedures," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 102(2), pages 167-178, April.
    • Handle: RePEc:spr:alstar:v:102:y:2018:i:2:d:10.1007_s10182-017-0297-0
    DOI: 10.1007/s10182-017-0297-0
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s10182-017-0297-0
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s10182-017-0297-0?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Axel Gandy & Georg Hahn, 2014. "MMCTest—A Safe Algorithm for Implementing Multiple Monte Carlo Tests," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 41(4), pages 1083-1101, December.
    2. Ajit C. Tamhane & Lingyun Liu, 2008. "On weighted Hochberg procedures," Biometrika, Biometrika Trust, vol. 95(2), pages 279-294.
    3. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    4. Gordon, Alexander Y. & Salzman, Peter, 2008. "Optimality of the Holm procedure among general step-down multiple testing procedures," Statistics & Probability Letters, Elsevier, vol. 78(13), pages 1878-1884, September.
    5. Guo Wenge & Peddada Shyamal, 2008. "Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-21, March.
    6. Gordon, Alexander Y., 2007. "Unimprovability of the Bonferroni procedure in the class of general step-up multiple testing procedures," Statistics & Probability Letters, Elsevier, vol. 77(2), pages 117-122, January.
    7. Axel Gandy & Georg Hahn, 2016. "A Framework for Monte Carlo based Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1046-1063, December.
    8. van Wieringen, Wessel N & van de Wiel, Mark A & van der Vaart, Aad W, 2008. "A Test for Partial Differential Expression," Journal of the American Statistical Association, American Statistical Association, vol. 103(483), pages 1039-1049.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Hahn, Georg, 2020. "On the expected runtime of multiple testing algorithms with bounded error," Statistics & Probability Letters, Elsevier, vol. 165(C).
    2. Gordon, Alexander Y., 2009. "Inequalities between generalized familywise error rates of a multiple testing procedure," Statistics & Probability Letters, Elsevier, vol. 79(19), pages 1996-2004, October.
    3. Dong Ding & Axel Gandy & Georg Hahn, 2020. "A simple method for implementing Monte Carlo tests," Computational Statistics, Springer, vol. 35(3), pages 1373-1392, September.
    4. Axel Gandy & Georg Hahn, 2016. "A Framework for Monte Carlo based Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 43(4), pages 1046-1063, December.
    5. Gordon, Alexander Y., 2012. "A sharp upper bound for the expected number of false rejections," Statistics & Probability Letters, Elsevier, vol. 82(8), pages 1507-1514.
    6. Fabian Kosse & Thomas Deckers & Pia Pinger & Hannah Schildberg-Hörisch & Armin Falk, 2020. "The Formation of Prosociality: Causal Evidence on the Role of Social Environment," Journal of Political Economy, University of Chicago Press, vol. 128(2), pages 434-467.
    7. Barrera, Oscar & Guriev, Sergei & Henry, Emeric & Zhuravskaya, Ekaterina, 2020. "Facts, alternative facts, and fact checking in times of post-truth politics," Journal of Public Economics, Elsevier, vol. 182(C).
    8. Hızıroğlu Aygün, Aysun & Kırdar, Murat Güray & Koyuncu, Murat & Stoeffler, Quentin, 2024. "Keeping refugee children in school and out of work: Evidence from the world's largest humanitarian cash transfer program," Journal of Development Economics, Elsevier, vol. 168(C).
    9. Cygan-Rehm, Kamila & Karbownik, Krzysztof, 2022. "The effects of incentivizing early prenatal care on infant health," Journal of Health Economics, Elsevier, vol. 83(C).
    10. Cindy Frascolla & Guillaume Lecuelle & Pascal Schlich & Hervé Cardot, 2022. "Two sample tests for Semi-Markov processes with parametric sojourn time distributions: an application in sensory analysis," Computational Statistics, Springer, vol. 37(5), pages 2553-2580, November.
    11. Agostinelli, Francesco & Avitabile, Ciro & Bobba, Matteo, 2021. "Enhancing Human Capital in Children: A Case Study on Scaling," TSE Working Papers 21-1196, Toulouse School of Economics (TSE), revised Oct 2023.
    12. Doyle, O. & Harmon, C. & Heckman, J.J. & Logue, C,; & Moon, S.H., 2013. "Measuring Investment in Human Capital Formation: An Experimental Analysis of Early Life Outcomes," Health, Econometrics and Data Group (HEDG) Working Papers 13/18, HEDG, c/o Department of Economics, University of York.
    13. Ann P. Bartel & Maya Rossin-Slater & Christopher J. Ruhm & Meredith Slopen & Jane Waldfogel, 2021. "The Impact of Paid Family Leave on Employers: Evidence from New York," NBER Working Papers 28672, National Bureau of Economic Research, Inc.
    14. Gabriella Conti & Christopher Hansman & James J. Heckman & Matthew F. X. Novak & Angela Ruggiero & Stephen J. Suomi, 2012. "Primate Evidence on the Late Health Effects of Early Life Adversity," Working Papers 2012-008, Human Capital and Economic Opportunity Working Group.
    15. Samuel Dodini, 2023. "Insurance Subsidies, the Affordable Care Act, and Financial Stability," Journal of Policy Analysis and Management, John Wiley & Sons, Ltd., vol. 42(1), pages 97-136, January.
    16. Clarke, Damian & Mühlrad, Hanna, 2021. "Abortion laws and women’s health," Journal of Health Economics, Elsevier, vol. 76(C).
    17. Krekel, Christian & De Neve, Jan-Emmanuel & Fancourt, Daisy & Layard, Richard, 2021. "A local community course that raises wellbeing and pro-sociality: Evidence from a randomised controlled trial," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 322-336.
    18. Gonzalo Nunez-Chaim & Henry G. Overman & Capucine Riom, 2024. "Does subsidising business advice improve firm performance? Evidence from a large RCT," CEP Discussion Papers dp1977, Centre for Economic Performance, LSE.
    19. Hvidman, Charlotte & Koch, Alexander K. & Nafziger, Julia & Nielsen, Søren Albeck & Rosholm, Michael, 2024. "An intensive, school-based learning camp targeting academic and non-cognitive skills evaluated in a randomized trial," Labour Economics, Elsevier, vol. 88(C).
    20. Capuno, Joseph & Kraft, Aleli & O'Donnell, Owen, 2021. "Effectiveness of clinic-based cardiovascular disease prevention: A randomized encouragement design experiment in the Philippines," Social Science & Medicine, Elsevier, vol. 283(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:alstar:v:102:y:2018:i:2:d:10.1007_s10182-017-0297-0. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.