IDEAS home Printed from https://ideas.repec.org/a/taf/jnlasa/v108y2013i504p1385-1401.html
   My bibliography  Save this article

Multiple Testing in a Two-Stage Adaptive Design With Combination Tests Controlling FDR

Author

Listed:
  • Sanat K. Sarkar
  • Jingjing Chen
  • Wenge Guo

Abstract

Testing multiple null hypotheses in two stages to decide which of these can be rejected or accepted at the first stage and which should be followed up for further testing having had additional observations is of importance in many scientific studies. We develop two procedures, each with two different combination functions, Fisher's and Simes', to combine p -values from two stages, given prespecified boundaries on the first-stage p -values in terms of the false discovery rate (FDR) and controlling the overall FDR at a desired level. The FDR control is proved when the pairs of first- and second-stage p -values are independent and those corresponding to the null hypotheses are identically distributed as a pair ( p 1 , p 2 ) satisfying the p -clud property. We did simulations to show that (1) our two-stage procedures can have significant power improvements over the first-stage Benjamini--Hochberg (BH) procedure compared to the improvement offered by the ideal BH procedure that one would have used had the second stage data been available for all the hypotheses, and can continue to control the FDR under some dependence situations, and (2) can offer considerable cost savings compared to the ideal BH procedure. The procedures are illustrated through a real gene expression data. Supplementary materials for this article are available online.

Suggested Citation

  • Sanat K. Sarkar & Jingjing Chen & Wenge Guo, 2013. "Multiple Testing in a Two-Stage Adaptive Design With Combination Tests Controlling FDR," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 108(504), pages 1385-1401, December.
    • Handle: RePEc:taf:jnlasa:v:108:y:2013:i:504:p:1385-1401
    DOI: 10.1080/01621459.2013.835662
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1080/01621459.2013.835662
    Download Restriction: Access to full text is restricted to subscribers.
    File URL: https://libkey.io/10.1080/01621459.2013.835662?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Brannath W. & Posch M. & Bauer P., 2002. "Recursive Combination Tests," Journal of the American Statistical Association, American Statistical Association, vol. 97, pages 236-244, March.
    2. Posch, Martin & Zehetmayer, Sonja & Bauer, Peter, 2009. "Hunting for Significance With the False Discovery Rate," Journal of the American Statistical Association, American Statistical Association, vol. 104(486), pages 832-840.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Ron Berman & Christophe Van den Bulte, 2022. "False Discovery in A/B Testing," Management Science, INFORMS, vol. 68(9), pages 6762-6782, September.
    2. Yi-Hui Zhou & Paul Brooks & Xiaoshan Wang, 2018. "A Two-Stage Hidden Markov Model Design for Biomarker Detection, with Application to Microbiome Research," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 10(1), pages 41-58, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Zehetmayer Sonja & Graf Alexandra C. & Posch Martin, 2015. "Sample size reassessment for a two-stage design controlling the false discovery rate," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 14(5), pages 429-442, November.
    2. Jingjing Chen, 2019. "A Note of Adaptive Design in Clinical Trials," Biostatistics and Biometrics Open Access Journal, Juniper Publishers Inc., vol. 9(5), pages 107-111, August.
    3. Georg Gutjahr & Werner Brannath & Peter Bauer, 2011. "An Approach to the Conditional Error Rate Principle with Nuisance Parameters," Biometrics, The International Biometric Society, vol. 67(3), pages 1039-1046, September.
    4. W. Brannath & P. Bauer & W. Maurer & M. Posch, 2003. "Sequential Tests for Noninferiority and Superiority," Biometrics, The International Biometric Society, vol. 59(1), pages 106-114, March.
    5. Rui Tang & Xiaoye Ma & Hui Yang & Michael Wolf, 2018. "Biomarker-Defined Subgroup Selection Adaptive Design for Phase III Confirmatory Trial with Time-to-Event Data: Comparing Group Sequential and Various Adaptive Enrichment Designs," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 10(2), pages 371-404, August.
    6. Rene Schmidt & Andreas Faldum & Robert Kwiecien, 2018. "Adaptive designs for the one†sample log†rank test," Biometrics, The International Biometric Society, vol. 74(2), pages 529-537, June.
    7. Nigel Stallard, 2023. "Adaptive enrichment designs with a continuous biomarker," Biometrics, The International Biometric Society, vol. 79(1), pages 9-19, March.
    8. Carl-Fredrik Burman & Christian Sonesson, 2006. "Are Flexible Designs Sound?," Biometrics, The International Biometric Society, vol. 62(3), pages 664-669, September.
    9. Parsons, Nick & Friede, Tim & Todd, Susan & Marquez, Elsa Valdes & Chataway, Jeremy & Nicholas, Richard & Stallard, Nigel, 2012. "An R package for implementing simulations for seamless phase II/III clinical trials using early outcomes for treatment selection," Computational Statistics & Data Analysis, Elsevier, vol. 56(5), pages 1150-1160.
    10. René Schmidt & Andreas Faldum & Joachim Gerß, 2015. "Adaptive designs with arbitrary dependence structure based on Fisher’s combination test," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(3), pages 427-447, September.
    11. P. Bauer, 2006. "Discussions," Biometrics, The International Biometric Society, vol. 62(3), pages 676-678, September.
    12. T. Tony Cai & Wenguang Sun, 2017. "Optimal screening and discovery of sparse signals with applications to multistage high throughput studies," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 79(1), pages 197-223, January.
    13. Werner Brannath & Cyrus R. Mehta & Martin Posch, 2009. "Exact Confidence Bounds Following Adaptive Group Sequential Tests," Biometrics, The International Biometric Society, vol. 65(2), pages 539-546, June.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:taf:jnlasa:v:108:y:2013:i:504:p:1385-1401. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Chris Longhurst (email available below). General contact details of provider: http://www.tandfonline.com/UASA20 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.