IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v134y2018icp29-35.html
   My bibliography  Save this article

The false discovery rate (FDR) of multiple tests in a class room lecture

Author

Listed:
  • Benditkis, Julia
  • Heesen, Philipp
  • Janssen, Arnold

Abstract

Multiple tests are designed to test a whole collection of null hypotheses simultaneously. Their quality is often judged by the false discovery rate (FDR), i.e. the expectation of the quotient of the number of false rejections divided by the number of all rejections. The widely cited Benjamini and Hochberg (BH) step up multiple test controls the FDR under various regularity assumptions. In this note we present a rapid approach to the BH step up and step down tests. Also, sharp FDR inequalities are discussed for dependent p-values and examples and counter-examples are considered. In particular, the Bonferroni bound is sharp under dependence for the control of the family-wise error rate.

Suggested Citation

  • Benditkis, Julia & Heesen, Philipp & Janssen, Arnold, 2018. "The false discovery rate (FDR) of multiple tests in a class room lecture," Statistics & Probability Letters, Elsevier, vol. 134(C), pages 29-35.
    • Handle: RePEc:eee:stapro:v:134:y:2018:i:c:p:29-35
    DOI: 10.1016/j.spl.2017.09.017
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167715217303115
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.spl.2017.09.017?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
    2. Guo, Wenge & Bhaskara Rao, M., 2008. "On optimality of the Benjamini-Hochberg procedure for the false discovery rate," Statistics & Probability Letters, Elsevier, vol. 78(14), pages 2024-2030, October.
    3. John D. Storey, 2002. "A direct approach to false discovery rates," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 64(3), pages 479-498, August.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Izmirlian, Grant, 2020. "Strong consistency and asymptotic normality for quantities related to the Benjamini–Hochberg false discovery rate procedure," Statistics & Probability Letters, Elsevier, vol. 160(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    2. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    3. Guo, Wenge & Bhaskara Rao, M., 2008. "On optimality of the Benjamini-Hochberg procedure for the false discovery rate," Statistics & Probability Letters, Elsevier, vol. 78(14), pages 2024-2030, October.
    4. Jeffrey T. Leek, 2011. "Asymptotic Conditional Singular Value Decomposition for High-Dimensional Genomic Data," Biometrics, The International Biometric Society, vol. 67(2), pages 344-352, June.
    5. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    6. Lianming Wang & David B. Dunson, 2010. "Semiparametric Bayes Multiple Testing: Applications to Tumor Data," Biometrics, The International Biometric Society, vol. 66(2), pages 493-501, June.
    7. Keith Cuthbertson & Dirk Nitzsche & Niall O'Sullivan, 2012. "False Discoveries in UK Mutual Fund Performance," European Financial Management, European Financial Management Association, vol. 18(3), pages 444-463, June.
    8. Hunt, Daniel L. & Cheng, Cheng & Pounds, Stanley, 2009. "The beta-binomial distribution for estimating the number of false rejections in microarray gene expression studies," Computational Statistics & Data Analysis, Elsevier, vol. 53(5), pages 1688-1700, March.
    9. Laurent Barras & Olivier Scaillet & Russ Wermers, 2010. "False Discoveries in Mutual Fund Performance: Measuring Luck in Estimated Alphas," Journal of Finance, American Finance Association, vol. 65(1), pages 179-216, February.
    10. Kim, Donggyu & Zhang, Chunming, 2014. "Adaptive linear step-up multiple testing procedure with the bias-reduced estimator," Statistics & Probability Letters, Elsevier, vol. 87(C), pages 31-39.
    11. Sermpinis, Georgios & Hassanniakalager, Arman & Stasinakis, Charalampos & Psaradellis, Ioannis, 2021. "Technical analysis profitability and Persistence: A discrete false discovery approach on MSCI indices," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 73(C).
    12. Psaradellis, Ioannis & Laws, Jason & Pantelous, Athanasios A. & Sermpinis, Georgios, 2023. "Technical analysis, spread trading, and data snooping control," International Journal of Forecasting, Elsevier, vol. 39(1), pages 178-191.
    13. Kang, Moonsu & Chun, Heuiju, 2011. "A generalized false discovery rate in microarray studies," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 731-737, January.
    14. Hongkai Ji & Wing Hung Wong, 2006. "Computational Biology: Toward Deciphering Gene Regulatory Information in Mammalian Genomes," Biometrics, The International Biometric Society, vol. 62(3), pages 645-663, September.
    15. Andrew Y. Chen, 2022. "Do t-Statistic Hurdles Need to be Raised?," Papers 2204.10275, arXiv.org, revised Apr 2024.
    16. Zhao, Haibing & Fung, Wing Kam, 2016. "A powerful FDR control procedure for multiple hypotheses," Computational Statistics & Data Analysis, Elsevier, vol. 98(C), pages 60-70.
    17. Lan, Wei & Zhong, Ping-Shou & Li, Runze & Wang, Hansheng & Tsai, Chih-Ling, 2016. "Testing a single regression coefficient in high dimensional linear models," Journal of Econometrics, Elsevier, vol. 195(1), pages 154-168.
    18. Xiaoquan Wen, 2017. "Robust Bayesian FDR Control Using Bayes Factors, with Applications to Multi-tissue eQTL Discovery," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(1), pages 28-49, June.
    19. Cai, Qingyun, 2018. "A scoring criterion for rejection of clustered p-values," Computational Statistics & Data Analysis, Elsevier, vol. 121(C), pages 180-189.
    20. Qingyun Cai & Hock Peng Chan, 2017. "A Double Application of the Benjamini-Hochberg Procedure for Testing Batched Hypotheses," Methodology and Computing in Applied Probability, Springer, vol. 19(2), pages 429-443, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:134:y:2018:i:c:p:29-35. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.