IDEAS home Printed from https://ideas.repec.org/p/bep/ucbbio/1140.html
   My bibliography  Save this paper

Multiple Testing. Part III. Procedures for Control of the Generalized Family-Wise Error Rate and Proportion of False Positives

Author

Listed:
  • Mark van der Laan

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • Sandrine Dudoit

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • Katherine Pollard

    (Center for Biomolecular Science & Engineering, University of California, Santa Cruz)

Abstract

The accompanying articles by Dudoit et al. (2003b) and van der Laan et al. (2003) provide single-step and step-down resampling-based multiple testing procedures that asymptotically control the family-wise error rate (FWER) for general null hypotheses and test statistics. The proposed procedures fundamentally differ from existing approaches in the choice of null distribution for deriving cut-offs for the test statistics and are shown to provide asymptotic control of the FWER under general data generating distributions, without the need for conditions such as subset pivotality. In this article, we show that any multiple testing procedure (asymptotically) controlling the FWER at level alpha can be augmented into: (i) a multiple testing procedure (asymptotically) controlling the generalized family-wise error rate (i.e., the probability, gFWER(k), of having more than k false positives) at level alpha and (ii) a multiple testing procedure (asymptotically) controlling the probability, PFP(q), that the proportion of false positives among the rejected hypotheses exceeds a user-supplied value q in (0,1) at level alpha. Existing procedures for control of the proportion of false positives typically rely on the assumption that the test statistics are independent, while our proposed augmentation procedures control the PFP and gFWER for general data generating distributions, with arbitrary dependence structures among variables. Applying our augmentation methods to step-down multiple testing procedures that asymptotically control the FWER at exact level alpha (van der Laan et al., 2003), yields multiple testing procedures that also asymptotically control the gFWER and PFP at exact level alpha. Finally, the adjusted p-values for the gFWER and PFP-controlling augmentation procedures are shown to be simple functions of the adjusted p-values for the original FWER-controlling procedure.

Suggested Citation

  • Mark van der Laan & Sandrine Dudoit & Katherine Pollard, 2004. "Multiple Testing. Part III. Procedures for Control of the Generalized Family-Wise Error Rate and Proportion of False Positives," U.C. Berkeley Division of Biostatistics Working Paper Series 1140, Berkeley Electronic Press.
    • Handle: RePEc:bep:ucbbio:1140
    Note: oai:bepress.com:ucbbiostat-1140
    as

    Download full text from publisher

    File URL: http://www.bepress.com/cgi/viewcontent.cgi?article=1140&context=ucbbiostat
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Efron B. & Tibshirani R. & Storey J.D. & Tusher V., 2001. "Empirical Bayes Analysis of a Microarray Experiment," Journal of the American Statistical Association, American Statistical Association, vol. 96, pages 1151-1160, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Youngchao Ge & Sandrine Dudoit & Terence Speed, 2003. "Resampling-based multiple testing for microarray data analysis," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 12(1), pages 1-77, June.
    2. Pounds Stanley B. & Gao Cuilan L. & Zhang Hui, 2012. "Empirical Bayesian Selection of Hypothesis Testing Procedures for Analysis of Sequence Count Expression Data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(5), pages 1-32, October.
    3. Niels Lundtorp Olsen & Alessia Pini & Simone Vantini, 2021. "False discovery rate for functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 784-809, September.
    4. Wen Shi & Xi Chen & Jennifer Shang, 2019. "An Efficient Morris Method-Based Framework for Simulation Factor Screening," INFORMS Journal on Computing, INFORMS, vol. 31(4), pages 745-770, October.
    5. Hossain, Ahmed & Beyene, Joseph & Willan, Andrew R. & Hu, Pingzhao, 2009. "A flexible approximate likelihood ratio test for detecting differential expression in microarray data," Computational Statistics & Data Analysis, Elsevier, vol. 53(10), pages 3685-3695, August.
    6. Dørum Guro & Snipen Lars & Solheim Margrete & Saebo Solve, 2011. "Smoothing Gene Expression Data with Network Information Improves Consistency of Regulated Genes," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-26, August.
    7. Daniel Yekutieli, 2015. "Bayesian tests for composite alternative hypotheses in cross-tabulated data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 24(2), pages 287-301, June.
    8. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    9. Ruth Heller & Saharon Rosset, 2021. "Optimal control of false discovery criteria in the two‐group model," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 83(1), pages 133-155, February.
    10. Yu Lianbo & Gulati Parul & Fernandez Soledad & Pennell Michael & Kirschner Lawrence & Jarjoura David, 2011. "Fully Moderated T-statistic for Small Sample Size Gene Expression Arrays," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 10(1), pages 1-22, September.
    11. Werft, W. & Benner, A. & Kopp-Schneider, A., 2012. "On the identification of predictive biomarkers: Detecting treatment-by-gene interaction in high-dimensional data," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1275-1286.
    12. repec:dau:papers:123456789/13437 is not listed on IDEAS
    13. Han, Shengtong & Zhang, Hongmei & Karmaus, Wilfried & Roberts, Graham & Arshad, Hasan, 2017. "Adjusting background noise in cluster analyses of longitudinal data," Computational Statistics & Data Analysis, Elsevier, vol. 109(C), pages 93-104.
    14. Ahmed Hossain & Hafiz T.A. Khan, 2016. "Identification of genomic markers correlated with sensitivity in solid tumors to Dasatinib using sparse principal components," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(14), pages 2538-2549, October.
    15. HyungJun Cho & Jaewoo Kang & Jae Lee, 2009. "Empirical Bayes analysis of unreplicated microarray data," Computational Statistics, Springer, vol. 24(3), pages 393-408, August.
    16. Xiaoquan Wen, 2017. "Robust Bayesian FDR Control Using Bayes Factors, with Applications to Multi-tissue eQTL Discovery," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(1), pages 28-49, June.
    17. Kline, Patrick & Walters, Christopher, 2019. "Audits as Evidence: Experiments, Ensembles, and Enforcement," Institute for Research on Labor and Employment, Working Paper Series qt3z72m9kn, Institute of Industrial Relations, UC Berkeley.
    18. He, Yi & Pan, Wei & Lin, Jizhen, 2006. "Cluster analysis using multivariate normal mixture models to detect differential gene expression with microarray data," Computational Statistics & Data Analysis, Elsevier, vol. 51(2), pages 641-658, November.
    19. Alejandro Ochoa & John D Storey & Manuel Llinás & Mona Singh, 2015. "Beyond the E-Value: Stratified Statistics for Protein Domain Prediction," PLOS Computational Biology, Public Library of Science, vol. 11(11), pages 1-21, November.
    20. Bansal Naveen K., 2007. "Decision theoretic Bayesian hypothesis testing with the selection goal," Statistics & Risk Modeling, De Gruyter, vol. 25(1), pages 19-39, January.
    21. Zhang Qi & Xu Zheng & Lai Yutong, 2021. "An Empirical Bayes approach for the identification of long-range chromosomal interaction from Hi-C data," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 20(1), pages 1-15, February.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bep:ucbbio:1140. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: http://www.bepress.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.