IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v77y2007i12p1362-1370.html
   My bibliography  Save this article

Simultaneous testing of multiple hypotheses using generalized p-values

Author

Listed:
  • Tsui, Kam-Wah
  • Tang, Shijie

Abstract

In the context of simultaneously testing many hypotheses, Benjamini and Hochberg [1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B (Methodological) 57, 289-300] propose a procedure that guarantees that the false discovery rate (FDR) will be less than or equal to a specified value. Here, the FDR is the expected value of the ratio of the number of incorrectly rejected hypotheses and the total number of rejected hypotheses. To our knowledge, all of the existing research assumes that a usual p-value is available for each hypothesis. However, in circumstances when nuisance parameters are present, the usual p-values may not be free of the nuisance parameters. We develop a simultaneous testing procedure to control the FDR for the problem of simultaneously testing many Behrens-Fisher problems. Our multiple testing procedure, based on generalized p-values [Tsui, K.-W., Weerahandi, S., 1989. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Assoc. 84(406), 602-607], is then illustrated with an application to data from a microarray experiment.

Suggested Citation

  • Tsui, Kam-Wah & Tang, Shijie, 2007. "Simultaneous testing of multiple hypotheses using generalized p-values," Statistics & Probability Letters, Elsevier, vol. 77(12), pages 1362-1370, July.
    • Handle: RePEc:eee:stapro:v:77:y:2007:i:12:p:1362-1370
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-7152(07)00100-9
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
    2. Tang, Shijie & Tsui, Kam-Wah, 2007. "Distributional properties for the generalized p-value for the Behrens-Fisher problem," Statistics & Probability Letters, Elsevier, vol. 77(1), pages 1-8, January.
    3. Genovese, Christopher R. & Wasserman, Larry, 2006. "Exceedance Control of the False Discovery Proportion," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1408-1417, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joshua Habiger & David Watts & Michael Anderson, 2017. "Multiple testing with heterogeneous multinomial distributions," Biometrics, The International Biometric Society, vol. 73(2), pages 562-570, June.
    2. Georgios Sermpinis & Arman Hassanniakalager & Charalampos Stasinakis & Ioannis Psaradellis, 2018. "Technical Analysis and Discrete False Discovery Rate: Evidence from MSCI Indices," Papers 1811.06766, arXiv.org, revised Jun 2019.
    3. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.
    4. Bajgrowicz, Pierre & Scaillet, Olivier, 2012. "Technical trading revisited: False discoveries, persistence tests, and transaction costs," Journal of Financial Economics, Elsevier, vol. 106(3), pages 473-491.
    5. Lan, Wei & Ding, Yue & Fang, Zheng & Fang, Kuangnan, 2016. "Testing covariates in high dimension linear regression with latent factors," Journal of Multivariate Analysis, Elsevier, vol. 144(C), pages 25-37.
    6. Zarrabi, Nima & Snaith, Stuart & Coakley, Jerry, 2017. "FX technical trading rules can be profitable sometimes!," International Review of Financial Analysis, Elsevier, vol. 49(C), pages 113-127.
    7. Gontscharuk, Veronika & Finner, Helmut, 2013. "Asymptotic FDR control under weak dependence: A counterexample," Statistics & Probability Letters, Elsevier, vol. 83(8), pages 1888-1893.
    8. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    9. Guo, Wenge & Bhaskara Rao, M., 2008. "On optimality of the Benjamini-Hochberg procedure for the false discovery rate," Statistics & Probability Letters, Elsevier, vol. 78(14), pages 2024-2030, October.
    10. Gharad Bryan & James J Choi & Dean Karlan, 2021. "Randomizing Religion: the Impact of Protestant Evangelism on Economic Outcomes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(1), pages 293-380.
    11. Jeffrey T. Leek, 2011. "Asymptotic Conditional Singular Value Decomposition for High-Dimensional Genomic Data," Biometrics, The International Biometric Society, vol. 67(2), pages 344-352, June.
    12. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    13. Lianming Wang & David B. Dunson, 2010. "Semiparametric Bayes Multiple Testing: Applications to Tumor Data," Biometrics, The International Biometric Society, vol. 66(2), pages 493-501, June.
    14. Hanck, Christoph, 2011. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," Economic Modelling, Elsevier, vol. 28(4), pages 1739-1746, July.
    15. Dette, Holger & Hoderlein, Stefan & Neumeyer, Natalie, 2016. "Testing multivariate economic restrictions using quantiles: The example of Slutsky negative semidefiniteness," Journal of Econometrics, Elsevier, vol. 191(1), pages 129-144.
    16. Keith Cuthbertson & Dirk Nitzsche & Niall O'Sullivan, 2012. "False Discoveries in UK Mutual Fund Performance," European Financial Management, European Financial Management Association, vol. 18(3), pages 444-463, June.
    17. Hunt, Daniel L. & Cheng, Cheng & Pounds, Stanley, 2009. "The beta-binomial distribution for estimating the number of false rejections in microarray gene expression studies," Computational Statistics & Data Analysis, Elsevier, vol. 53(5), pages 1688-1700, March.
    18. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    19. Döhler, Sebastian, 2018. "A discrete modification of the Benjamini–Yekutieli procedure," Econometrics and Statistics, Elsevier, vol. 5(C), pages 137-147.
    20. Dean Palejev & Mladen Savov, 2021. "On the Convergence of the Benjamini–Hochberg Procedure," Mathematics, MDPI, vol. 9(17), pages 1-19, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:77:y:2007:i:12:p:1362-1370. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.