IDEAS home Printed from https://ideas.repec.org/a/bpj/sagmbi/v3y2004i1n13.html
   My bibliography  Save this article

Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates

Author

Listed:
  • Dudoit Sandrine

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • van der Laan Mark J.

    (Division of Biostatistics, School of Public Health, University of California, Berkeley)

  • Pollard Katherine S.

    (University of California, Santa Cruz)

Abstract

The present article proposes general single-step multiple testing procedures for controlling Type I error rates defined as arbitrary parameters of the distribution of the number of Type I errors, such as the generalized family-wise error rate. A key feature of our approach is the test statistics null distribution (rather than data generating null distribution) used to derive cut-offs (i.e., rejection regions) for these test statistics and the resulting adjusted p-values. For general null hypotheses, corresponding to submodels for the data generating distribution, we identify an asymptotic domination condition for a null distribution under which single-step common-quantile and common-cut-off procedures asymptotically control the Type I error rate, for arbitrary data generating distributions, without the need for conditions such as subset pivotality. Inspired by this general characterization of a null distribution, we then propose as an explicit null distribution the asymptotic distribution of the vector of null value shifted and scaled test statistics. In the special case of family-wise error rate (FWER) control, our method yields the single-step minP and maxT procedures, based on minima of unadjusted p-values and maxima of test statistics, respectively, with the important distinction in the choice of null distribution. Single-step procedures based on consistent estimators of the null distribution are shown to also provide asymptotic control of the Type I error rate. A general bootstrap algorithm is supplied to conveniently obtain consistent estimators of the null distribution. The special cases of t- and F-statistics are discussed in detail. The companion articles focus on step-down multiple testing procedures for control of the FWER (van der Laan et al., 2004b) and on augmentations of FWER-controlling methods to control error rates such as tail probabilities for the number of false positives and for the proportion of false positives among the rejected hypotheses (van der Laan et al., 2004a). The proposed bootstrap multiple testing procedures are evaluated by a simulation study and applied to genomic data in the fourth article of the series (Pollard et al., 2004).

Suggested Citation

  • Dudoit Sandrine & van der Laan Mark J. & Pollard Katherine S., 2004. "Multiple Testing. Part I. Single-Step Procedures for Control of General Type I Error Rates," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 3(1), pages 1-71, June.
    • Handle: RePEc:bpj:sagmbi:v:3:y:2004:i:1:n:13
    DOI: 10.2202/1544-6115.1040
    as

    Download full text from publisher

    File URL: https://doi.org/10.2202/1544-6115.1040
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.2202/1544-6115.1040?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Miecznikowski, Jeffrey C. & Gold, David & Shepherd, Lori & Liu, Song, 2011. "Deriving and comparing the distribution for the number of false positives in single step methods to control k-FWER," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1695-1705, November.
    2. Joseph P. Romano & Michael Wolf, 2008. "Balanced Control of Generalized Error Rates," IEW - Working Papers 379, Institute for Empirical Research in Economics - University of Zurich.
    3. Rubin Daniel & Dudoit Sandrine & van der Laan Mark, 2006. "A Method to Increase the Power of Multiple Testing Procedures Through Sample Splitting," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 5(1), pages 1-20, August.
    4. Yoshiyuki Ninomiya & Hironori Fujisawa, 2007. "A Conservative Test for Multiple Comparison Based on Highly Correlated Test Statistics," Biometrics, The International Biometric Society, vol. 63(4), pages 1135-1142, December.
    5. Hossain, Ahmed & Beyene, Joseph & Willan, Andrew R. & Hu, Pingzhao, 2009. "A flexible approximate likelihood ratio test for detecting differential expression in microarray data," Computational Statistics & Data Analysis, Elsevier, vol. 53(10), pages 3685-3695, August.
    6. Merrill Birkner & Sandra Sinisi & Mark van der Laan, 2004. "Multiple Testing and Data Adaptive Regression: An Application to HIV-1 Sequence Data," U.C. Berkeley Division of Biostatistics Working Paper Series 1161, Berkeley Electronic Press.
    7. Christina C. Bartenschlager & Michael Krapp, 2015. "Theorie und Methoden multipler statistischer Vergleiche," AStA Wirtschafts- und Sozialstatistisches Archiv, Springer;Deutsche Statistische Gesellschaft - German Statistical Society, vol. 9(2), pages 107-129, November.
    8. Miecznikowski Jeffrey C. & Gaile Daniel P., 2014. "A novel characterization of the generalized family wise error rate using empirical null distributions," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 13(3), pages 299-322, June.
    9. van der Laan Mark J. & Hubbard Alan E., 2006. "Quantile-Function Based Null Distribution in Resampling Based Multiple Testing," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 5(1), pages 1-30, May.
    10. Schumi Jennifer & DiRienzo A. Gregory & DeGruttola Victor, 2008. "Testing for Associations with Missing High-Dimensional Categorical Covariates," The International Journal of Biostatistics, De Gruyter, vol. 4(1), pages 1-19, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:sagmbi:v:3:y:2004:i:1:n:13. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.