IDEAS home Printed from https://ideas.repec.org/p/bep/ucbbio/1138.html
   My bibliography  Save this paper

Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error Rate

Author

Listed:
  • Mark van der Laan

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • Sandrine Dudoit

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • Katherine Pollard

    (Center for Biomolecular Science & Engineering, University of California, Santa Cruz)

Abstract

The present article proposes two step-down multiple testing procedures for asymptotic control of the family-wise error rate (FWER): the first procedure is based on maxima of test statistics (step-down maxT), while the second relies on minima of unadjusted p-values (step-down minP). A key feature of our approach is the test statistics null distribution (rather than data generating null distribution) used to derive cut-offs (i.e., rejection regions) for these test statistics and the resulting adjusted p-values. For general null hypotheses, corresponding to submodels for the data generating distribution, we identify an asymptotic domination condition for a null distribution under which the step-down maxT and minP procedures asymptotically control the Type I error rate, for arbitrary data generating distributions, without the need for conditions such as subset pivotality. Inspired by this general characterization of a null distribution, we then propose as an explicit null distribution the asymptotic distribution of the vector of null-value shifted and scaled test statistics. Step-down procedures based on consistent estimators of the null distribution are shown to also provide asymptotic control of the Type I error rate. A general bootstrap algorithm is supplied to conveniently obtain consistent estimators of the null distribution.

Suggested Citation

  • Mark van der Laan & Sandrine Dudoit & Katherine Pollard, 2004. "Multiple Testing. Part II. Step-Down Procedures for Control of the Family-Wise Error Rate," U.C. Berkeley Division of Biostatistics Working Paper Series 1138, Berkeley Electronic Press.
    • Handle: RePEc:bep:ucbbio:1138
    Note: oai:bepress.com:ucbbiostat-1138
    as

    Download full text from publisher

    File URL: http://www.bepress.com/cgi/viewcontent.cgi?article=1138&context=ucbbiostat
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Youngchao Ge & Sandrine Dudoit & Terence Speed, 2003. "Resampling-based multiple testing for microarray data analysis," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 12(1), pages 1-77, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Áureo de Paula & Xun Tang, 2012. "Inference of Signs of Interaction Effects in Simultaneous Games With Incomplete Information," Econometrica, Econometric Society, vol. 80(1), pages 143-172, January.
    2. Aureo de Paula & Xun Tang, 2010. "Inference of Signs of Interaction Effects in Simultaneous Games with Incomplete Information, Second Version," PIER Working Paper Archive 11-003, Penn Institute for Economic Research, Department of Economics, University of Pennsylvania, revised 12 Feb 2011.
    3. Yang Yang & Victor DeGruttola, 2008. "Resampling-Based Multiple Testing Methods with Covariate Adjustment: Application to Investigation of Antiretroviral Drug Susceptibility," Biometrics, The International Biometric Society, vol. 64(2), pages 329-336, June.
    4. Schumi Jennifer & DiRienzo A. Gregory & DeGruttola Victor, 2008. "Testing for Associations with Missing High-Dimensional Categorical Covariates," The International Journal of Biostatistics, De Gruyter, vol. 4(1), pages 1-19, September.
    5. Alessio Farcomeni, 2009. "Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(3), pages 501-517, September.
    6. Miecznikowski, Jeffrey C. & Gold, David & Shepherd, Lori & Liu, Song, 2011. "Deriving and comparing the distribution for the number of false positives in single step methods to control k-FWER," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1695-1705, November.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ebrahimi, Nader, 2008. "Simultaneous control of false positives and false negatives in multiple hypotheses testing," Journal of Multivariate Analysis, Elsevier, vol. 99(3), pages 437-450, March.
    2. Pittelkow Yvonne E & Wilson Susan R, 2003. "Visualisation of Gene Expression Data - the GE-biplot, the Chip-plot and the Gene-plot," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 2(1), pages 1-19, September.
    3. Bergamelli, Michele & Bianchi, Annamaria & Khalaf, Lynda & Urga, Giovanni, 2019. "Combining p-values to test for multiple structural breaks in cointegrated regressions," Journal of Econometrics, Elsevier, vol. 211(2), pages 461-482.
    4. Debashis Ghosh, 2006. "Shrunken p-Values for Assessing Differential Expression with Applications to Genomic Data Analysis," Biometrics, The International Biometric Society, vol. 62(4), pages 1099-1106, December.
    5. Ge, Yongchao & Sealfon, Stuart C. & Tseng, Chi-Hong & Speed, Terence P., 2007. "A Holm-type procedure controlling the false discovery rate," Statistics & Probability Letters, Elsevier, vol. 77(18), pages 1756-1762, December.
    6. Baolin Wu & Zhong Guan & Hongyu Zhao, 2006. "Parametric and Nonparametric FDR Estimation Revisited," Biometrics, The International Biometric Society, vol. 62(3), pages 735-744, September.
    7. Kai Yu & William Wheeler & Qizhai Li & Andrew W. Bergen & Neil Caporaso & Nilanjan Chatterjee & Jinbo Chen, 2010. "A Partially Linear Tree-based Regression Model for Multivariate Outcomes," Biometrics, The International Biometric Society, vol. 66(1), pages 89-96, March.
    8. Lippmann, Quentin, 2022. "Gender and lawmaking in times of quotas," Journal of Public Economics, Elsevier, vol. 207(C).
    9. Xuesong Yu & Timothy W. Randolph & Hua Tang & Li Hsu, 2010. "Detecting Genomic Aberrations Using Products in a Multiscale Analysis," Biometrics, The International Biometric Society, vol. 66(3), pages 684-693, September.
    10. Fu, Hsuan & Luger, Richard, 2022. "Multiple testing of the forward rate unbiasedness hypothesis across currencies," Journal of Empirical Finance, Elsevier, vol. 68(C), pages 232-245.
    11. Edward L. Korn & Boris Freidlin, 2008. "A Note on Controlling the Number of False Positives," Biometrics, The International Biometric Society, vol. 64(1), pages 227-231, March.
    12. Tsai, Chen-An & Chen, James J., 2007. "Kernel estimation for adjusted p-values in multiple testing," Computational Statistics & Data Analysis, Elsevier, vol. 51(8), pages 3885-3897, May.
    13. Guo Wenge & Peddada Shyamal, 2008. "Adaptive Choice of the Number of Bootstrap Samples in Large Scale Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-21, March.
    14. Dazard, Jean-Eudes & Sunil Rao, J., 2012. "Joint adaptive mean–variance regularization and variance stabilization of high dimensional data," Computational Statistics & Data Analysis, Elsevier, vol. 56(7), pages 2317-2333.
    15. Wu, Jincao & Patwa, Tasneem H. & Lubman, David M. & Ghosh, Debashis, 2009. "Identification of differentially expressed spatial clusters using humoral response microarray data," Computational Statistics & Data Analysis, Elsevier, vol. 53(8), pages 3094-3102, June.
    16. Zhenchuan Wang & Qiuying Sha & Shuanglin Zhang, 2016. "Joint Analysis of Multiple Traits Using "Optimal" Maximum Heritability Test," PLOS ONE, Public Library of Science, vol. 11(3), pages 1-12, March.
    17. Richard Luger, 2024. "Regularizing stock return covariance matrices via multiple testing of correlations," Papers 2407.09696, arXiv.org.
    18. Bickel David R., 2008. "Correcting the Estimated Level of Differential Expression for Gene Selection Bias: Application to a Microarray Study," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-27, March.
    19. Yifan Gu & Yang Qi & Pulin Gong, 2019. "Rich-club connectivity, diverse population coupling, and dynamical activity patterns emerging from local cortical circuits," PLOS Computational Biology, Public Library of Science, vol. 15(4), pages 1-34, April.
    20. Jesse Hemerik & Jelle Goeman, 2018. "Exact testing with random permutations," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(4), pages 811-825, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bep:ucbbio:1138. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Christopher F. Baum (email available below). General contact details of provider: http://www.bepress.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.