IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v79y2023i4p2781-2793.html
   My bibliography  Save this article

Optimal test procedures for multiple hypotheses controlling the familywise expected loss

Author

Listed:
  • Willi Maurer
  • Frank Bretz
  • Xiaolei Xun

Abstract

We consider the problem of testing multiple null hypotheses, where a decision to reject or retain must be made for each one and embedding incorrect decisions into a real‐life context may inflict different losses. We argue that traditional methods controlling the Type I error rate may be too restrictive in this situation and that the standard familywise error rate may not be appropriate. Using a decision‐theoretic approach, we define suitable loss functions for a given decision rule, where incorrect decisions can be treated unequally by assigning different loss values. Taking expectation with respect to the sampling distribution of the data allows us to control the familywise expected loss instead of the conventional familywise error rate. Different loss functions can be adopted, and we search for decision rules that satisfy certain optimality criteria within a broad class of decision rules for which the expected loss is bounded by a fixed threshold under any parameter configuration. We illustrate the methods with the problem of establishing efficacy of a new medicinal treatment in non‐overlapping subgroups of patients.

Suggested Citation

  • Willi Maurer & Frank Bretz & Xiaolei Xun, 2023. "Optimal test procedures for multiple hypotheses controlling the familywise expected loss," Biometrics, The International Biometric Society, vol. 79(4), pages 2781-2793, December.
  • Handle: RePEc:bla:biomet:v:79:y:2023:i:4:p:2781-2793
    DOI: 10.1111/biom.13907
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.13907
    Download Restriction: no

    File URL: https://libkey.io/10.1111/biom.13907?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Peter Muller & Giovanni Parmigiani & Christian Robert & Judith Rousseau, 2004. "Optimal Sample Size for Multiple Testing: The Case of Gene Expression Microarrays," Journal of the American Statistical Association, American Statistical Association, vol. 99, pages 990-1001, December.
    2. Michael Rosenblum & Han Liu & En-Hsu Yen, 2014. "Optimal Tests of Treatment Effects for the Overall Population and Two Subpopulations in Randomized Trials, Using Sparse Linear Programming," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 109(507), pages 1216-1228, September.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Werner Brannath, 2023. "Discussion on “Optimal test procedures for multiple hypotheses controlling the familywise expected loss” by Willi Maurer, Frank Bretz, and Xiaolei Xun," Biometrics, The International Biometric Society, vol. 79(4), pages 2806-2810, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ghosh Debashis, 2012. "Incorporating the Empirical Null Hypothesis into the Benjamini-Hochberg Procedure," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 11(4), pages 1-21, July.
    2. Gómez-Villegas Miguel A. & Salazar Isabel & Sanz Luis, 2014. "A Bayesian decision procedure for testing multiple hypotheses in DNA microarray experiments," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 13(1), pages 49-65, February.
    3. Edsel Peña & Joshua Habiger & Wensong Wu, 2015. "Classes of multiple decision functions strongly controlling FWER and FDR," Metrika: International Journal for Theoretical and Applied Statistics, Springer, vol. 78(5), pages 563-595, July.
    4. Xiaoquan Wen, 2017. "Robust Bayesian FDR Control Using Bayes Factors, with Applications to Multi-tissue eQTL Discovery," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 9(1), pages 28-49, June.
    5. Zichen Ma & Shannon W. Davis & Yen‐Yi Ho, 2023. "Flexible copula model for integrating correlated multi‐omics data from single‐cell experiments," Biometrics, The International Biometric Society, vol. 79(2), pages 1559-1572, June.
    6. Noirrit Kiran Chandra & Sourabh Bhattacharya, 2021. "Asymptotic theory of dependent Bayesian multiple testing procedures under possible model misspecification," Annals of the Institute of Statistical Mathematics, Springer;The Institute of Statistical Mathematics, vol. 73(5), pages 891-920, October.
    7. Sudipto Banerjee, 2023. "Discussion of “Optimal test procedures for multiple hypotheses controlling the familywise expected loss” by Willi Maurer, Frank Bretz, and Xiaolei Xun," Biometrics, The International Biometric Society, vol. 79(4), pages 2798-2801, December.
    8. Yoav Benjamini & Ruth Heller & Abba Krieger & Saharon Rosset, 2023. "Discussion on “Optimal test procedures for multiple hypotheses controlling the familywise expected loss” by Willi Maurer, Frank Bretz, and Xiaolei Xun," Biometrics, The International Biometric Society, vol. 79(4), pages 2794-2797, December.
    9. Willi Maurer & Frank Bretz & Xiaolei Xun, 2023. "Rejoinder to discussions on “Optimal test procedures for multiple hypotheses controlling the familywise expected loss”," Biometrics, The International Biometric Society, vol. 79(4), pages 2811-2814, December.
    10. Kiranmoy Das, 2016. "A semiparametric Bayesian approach for joint modeling of longitudinal trait and event time," Journal of Applied Statistics, Taylor & Francis Journals, vol. 43(15), pages 2850-2865, November.
    11. Michael Rosenblum & Ethan X. Fang & Han Liu, 2020. "Optimal, two‐stage, adaptive enrichment designs for randomized trials, using sparse linear programming," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 749-772, July.
    12. Jack Jewson & Li Li & Laura Battaglia & Stephen Hansen & David Rossell & Piotr Zwiernik, 2022. "Graphical model inference with external network data," CeMMAP working papers 20/22, Institute for Fiscal Studies.
    13. Michele Guindani & Wesley O. Johnson, 2018. "More nonparametric Bayesian inference in applications," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 27(2), pages 239-251, June.
    14. Cheng‐Han Yu & Raquel Prado & Hernando Ombao & Daniel Rowe, 2023. "Bayesian spatiotemporal modeling on complex‐valued fMRI signals via kernel convolutions," Biometrics, The International Biometric Society, vol. 79(2), pages 616-628, June.
    15. Elisa C. J. Maria & Isabel Salazar & Luis Sanz & Miguel A. Gómez-Villegas, 2020. "Using Copula to Model Dependence When Testing Multiple Hypotheses in DNA Microarray Experiments: A Bayesian Approximation," Mathematics, MDPI, vol. 8(9), pages 1-22, September.
    16. Ilana Belitskaya-Lévy & Hui Wang & Mei-Chiung Shih & Lu Tian & Gheorghe Doros & Robert A. Lew & Ying Lu, 2018. "A New Overall-Subgroup Simultaneous Test for Optimal Inference in Biomarker-Targeted Confirmatory Trials," Statistics in Biosciences, Springer;International Chinese Statistical Association, vol. 10(2), pages 297-323, August.
    17. David Lähnemann & Johannes Köster & Ute Fischer & Arndt Borkhardt & Alice C. McHardy & Alexander Schönhuth, 2021. "Accurate and scalable variant calling from single cell DNA sequencing data with ProSolo," Nature Communications, Nature, vol. 12(1), pages 1-11, December.
    18. Rossell David & Guerra Rudy & Scott Clayton, 2008. "Semi-Parametric Differential Expression Analysis via Partial Mixture Estimation," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 7(1), pages 1-29, April.
    19. Zhaoyang Tian & Kun Liang & Pengfei Li, 2021. "A powerful procedure that controls the false discovery rate with directional information," Biometrics, The International Biometric Society, vol. 77(1), pages 212-222, March.
    20. Dickhaus Thorsten, 2015. "Simultaneous Bayesian analysis of contingency tables in genetic association studies," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 14(4), pages 347-360, August.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:79:y:2023:i:4:p:2781-2793. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.