IDEAS home Printed from https://ideas.repec.org/a/bpj/sagmbi/v6y2007i1n3.html
   My bibliography  Save this article

A Generalized Sidak-Holm Procedure and Control of Generalized Error Rates under Independence

Author

Listed:
  • Guo Wenge

    (University of Cincinnati)

  • Romano Joseph

    (Stanford University)

Abstract

For testing multiple null hypotheses, the classical approach to dealing with the multiplicity problem is to restrict attention to procedures that control the familywise error rate (FWER), the probability of even one false rejection. In many applications, one might be willing to tolerate more than one false rejection provided the number of such cases is controlled, thereby increasing the ability of the procedure to detect false null hypotheses. This suggests replacing control of the FWER by controlling the probability of k or more false rejections, which is called the k-FWER. In Hommel and Hoffmann (1987) and Lehmann and Romano (2005a), single step and stepdown procedures are derived that control the k-FWER, without making any assumptions concerning the dependence structure of the p-values of the individual tests. However, if the p-values are mutually independent, one can improve the procedures. In fact, Sarkar (2005) provided such an improvement. However, we show other improvements are possible which appear to be generally much better, and are sometimes unimprovable. When k=1, the procedure reduces to the classical method of Sidak, and the stepdown procedure is unimprovable and strictly dominates that of Sarkar. Under a monotonicity condition, an unimprovable procedure is obtained. In the case k=2, the monotonicity condition is satisfied, and the condition can be checked numerically in general. We then develop a stepdown method that controls the false discovery proportion. Except for the case of k-FWER control with k=1, the gains are surprisingly dramatic, and theoretical and numerical evidence is given.

Suggested Citation

  • Guo Wenge & Romano Joseph, 2007. "A Generalized Sidak-Holm Procedure and Control of Generalized Error Rates under Independence," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 6(1), pages 1-35, January.
  • Handle: RePEc:bpj:sagmbi:v:6:y:2007:i:1:n:3
    DOI: 10.2202/1544-6115.1247
    as

    Download full text from publisher

    File URL: https://doi.org/10.2202/1544-6115.1247
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.

    File URL: https://libkey.io/10.2202/1544-6115.1247?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pallavi Basu & Luella Fu & Alessio Saretto & Wenguang Sun, 2021. "Empirical Bayes Control of the False Discovery Exceedance," Working Papers 2115, Federal Reserve Bank of Dallas.
    2. Alessio Farcomeni, 2009. "Generalized Augmentation to Control the False Discovery Exceedance in Multiple Testing," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 36(3), pages 501-517, September.
    3. Miecznikowski, Jeffrey C. & Gold, David & Shepherd, Lori & Liu, Song, 2011. "Deriving and comparing the distribution for the number of false positives in single step methods to control k-FWER," Statistics & Probability Letters, Elsevier, vol. 81(11), pages 1695-1705, November.
    4. Martin, Ian W.R. & Nagel, Stefan, 2022. "Market efficiency in the age of big data," Journal of Financial Economics, Elsevier, vol. 145(1), pages 154-177.
    5. Edward L. Korn & Boris Freidlin, 2008. "A Note on Controlling the Number of False Positives," Biometrics, The International Biometric Society, vol. 64(1), pages 227-231, March.
    6. Li Wang, 2019. "Weighted multiple testing procedure for grouped hypotheses with k-FWER control," Computational Statistics, Springer, vol. 34(2), pages 885-909, June.
    7. Cerioli, Andrea & Farcomeni, Alessio, 2011. "Error rates for multivariate outlier detection," Computational Statistics & Data Analysis, Elsevier, vol. 55(1), pages 544-553, January.
    8. Wang, Li, 2022. "New testing procedures with k-FWER control for discrete data," Statistics & Probability Letters, Elsevier, vol. 180(C).
    9. L. Finos & A. Farcomeni, 2011. "k-FWER Control without p -value Adjustment, with Application to Detection of Genetic Determinants of Multiple Sclerosis in Italian Twins," Biometrics, The International Biometric Society, vol. 67(1), pages 174-181, March.
    10. Wang, Li & Xu, Xingzhong, 2012. "Step-up procedure controlling generalized family-wise error rate," Statistics & Probability Letters, Elsevier, vol. 82(4), pages 775-782.
    11. Kramer, Dennis A. & Lamb, Christina & Page, Lindsay C., 2021. "The effects of default choice on student loan borrowing: Experimental evidence from a public research university," Journal of Economic Behavior & Organization, Elsevier, vol. 189(C), pages 470-489.
    12. Miecznikowski Jeffrey C. & Gaile Daniel P., 2014. "A novel characterization of the generalized family wise error rate using empirical null distributions," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 13(3), pages 299-322, June.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:sagmbi:v:6:y:2007:i:1:n:3. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    We have no bibliographic references for this item. You can help adding them by using this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.