IDEAS home Printed from https://ideas.repec.org/a/eee/stapro/v77y2007i12p1362-1370.html
   My bibliography  Save this article

Simultaneous testing of multiple hypotheses using generalized p-values

Author

Listed:
  • Tsui, Kam-Wah
  • Tang, Shijie

Abstract

In the context of simultaneously testing many hypotheses, Benjamini and Hochberg [1995. Controlling the false discovery rate: a practical and powerful approach to multiple testing. J. Roy. Statist. Soc. Ser. B (Methodological) 57, 289-300] propose a procedure that guarantees that the false discovery rate (FDR) will be less than or equal to a specified value. Here, the FDR is the expected value of the ratio of the number of incorrectly rejected hypotheses and the total number of rejected hypotheses. To our knowledge, all of the existing research assumes that a usual p-value is available for each hypothesis. However, in circumstances when nuisance parameters are present, the usual p-values may not be free of the nuisance parameters. We develop a simultaneous testing procedure to control the FDR for the problem of simultaneously testing many Behrens-Fisher problems. Our multiple testing procedure, based on generalized p-values [Tsui, K.-W., Weerahandi, S., 1989. Generalized p-values in significance testing of hypotheses in the presence of nuisance parameters. J. Amer. Statist. Assoc. 84(406), 602-607], is then illustrated with an application to data from a microarray experiment.

Suggested Citation

  • Tsui, Kam-Wah & Tang, Shijie, 2007. "Simultaneous testing of multiple hypotheses using generalized p-values," Statistics & Probability Letters, Elsevier, vol. 77(12), pages 1362-1370, July.
    • Handle: RePEc:eee:stapro:v:77:y:2007:i:12:p:1362-1370
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-7152(07)00100-9
    Download Restriction: Full text for ScienceDirect subscribers only
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Genovese, Christopher R. & Wasserman, Larry, 2006. "Exceedance Control of the False Discovery Proportion," Journal of the American Statistical Association, American Statistical Association, vol. 101, pages 1408-1417, December.
    2. Tang, Shijie & Tsui, Kam-Wah, 2007. "Distributional properties for the generalized p-value for the Behrens-Fisher problem," Statistics & Probability Letters, Elsevier, vol. 77(1), pages 1-8, January.
    3. John D. Storey & Jonathan E. Taylor & David Siegmund, 2004. "Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 66(1), pages 187-205, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Habiger, Joshua D. & Peña, Edsel A., 2014. "Compound p-value statistics for multiple testing procedures," Journal of Multivariate Analysis, Elsevier, vol. 126(C), pages 153-166.
    2. Joshua Habiger & David Watts & Michael Anderson, 2017. "Multiple testing with heterogeneous multinomial distributions," Biometrics, The International Biometric Society, vol. 73(2), pages 562-570, June.
    3. Georgios Sermpinis & Arman Hassanniakalager & Charalampos Stasinakis & Ioannis Psaradellis, 2018. "Technical Analysis and Discrete False Discovery Rate: Evidence from MSCI Indices," Papers 1811.06766, arXiv.org, revised Jun 2019.
    4. Jianqing Fan & Xu Han, 2017. "Estimation of the false discovery proportion with unknown dependence," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(4), pages 1143-1164, September.
    5. Shigeyuki Matsui & Hisashi Noma, 2011. "Estimating Effect Sizes of Differentially Expressed Genes for Power and Sample-Size Assessments in Microarray Experiments," Biometrics, The International Biometric Society, vol. 67(4), pages 1225-1235, December.
    6. Lianming Wang & David B. Dunson, 2010. "Semiparametric Bayes Multiple Testing: Applications to Tumor Data," Biometrics, The International Biometric Society, vol. 66(2), pages 493-501, June.
    7. Hanck, Christoph, 2011. "Now, whose schools are really better (or weaker) than Germany's? A multiple testing approach," Economic Modelling, Elsevier, vol. 28(4), pages 1739-1746, July.
    8. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    9. Dean Palejev & Mladen Savov, 2021. "On the Convergence of the Benjamini–Hochberg Procedure," Mathematics, MDPI, vol. 9(17), pages 1-19, September.
    10. Moon, H.R. & Perron, B., 2012. "Beyond panel unit root tests: Using multiple testing to determine the nonstationarity properties of individual series in a panel," Journal of Econometrics, Elsevier, vol. 169(1), pages 29-33.
    11. Chen, Song Xi & Guo, Bin & Qiu, Yumou, 2023. "Testing and signal identification for two-sample high-dimensional covariances via multi-level thresholding," Journal of Econometrics, Elsevier, vol. 235(2), pages 1337-1354.
    12. Psaradellis, Ioannis & Laws, Jason & Pantelous, Athanasios A. & Sermpinis, Georgios, 2023. "Technical analysis, spread trading, and data snooping control," International Journal of Forecasting, Elsevier, vol. 39(1), pages 178-191.
    13. G�nther Fink & Margaret McConnell & Sebastian Vollmer, 2014. "Testing for heterogeneous treatment effects in experimental data: false discovery risks and correction procedures," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(1), pages 44-57, January.
    14. repec:hum:wpaper:sfb649dp2012-049 is not listed on IDEAS
    15. Dette, Holger & Hoderlein, Stefan & Neumeyer, Natalie, 2016. "Testing multivariate economic restrictions using quantiles: The example of Slutsky negative semidefiniteness," Journal of Econometrics, Elsevier, vol. 191(1), pages 129-144.
    16. Gharad Bryan & James J Choi & Dean Karlan, 2021. "Randomizing Religion: the Impact of Protestant Evangelism on Economic Outcomes," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 136(1), pages 293-380.
    17. Wang Chamont & Gevertz Jana L., 2016. "Finding causative genes from high-dimensional data: an appraisal of statistical and machine learning approaches," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 15(4), pages 321-347, August.
    18. de Uña-Alvarez Jacobo, 2011. "On the Statistical Properties of SGoF Multitesting Method," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-30, April.
    19. Ferreira José A. & Berkhof Johannes & Souverein Olga & Zwinderman Koos, 2009. "A Multiple Testing Approach to High-Dimensional Association Studies with an Application to the Detection of Associations between Risk Factors of Heart Disease and Genetic Polymorphisms," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 8(1), pages 1-58, January.
    20. Huang, Rong & Pilbeam, Keith & Pouliot, William, 2021. "Do actively managed US mutual funds produce positive alpha?," Journal of Economic Behavior & Organization, Elsevier, vol. 182(C), pages 472-492.
    21. Baolin Wu & Zhong Guan & Hongyu Zhao, 2006. "Parametric and Nonparametric FDR Estimation Revisited," Biometrics, The International Biometric Society, vol. 62(3), pages 735-744, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:stapro:v:77:y:2007:i:12:p:1362-1370. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/wps/find/journaldescription.cws_home/622892/description#description .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.