IDEAS home Printed from https://ideas.repec.org/a/oup/biomet/v111y2024i2p441-458..html
   My bibliography  Save this article

More efficient exact group invariance testing: using a representative subgroup

Author

Listed:
  • N W Koning
  • J Hemerik

Abstract

SummaryWe consider testing invariance of a distribution under an algebraic group of transformations, such as permutations or sign flips. As such groups are typically huge, tests based on the full group are often computationally infeasible. Hence, it is standard practice to use a random subset of transformations. We improve upon this by replacing the random subset with a strategically chosen, fixed subgroup of transformations. In a generalized location model, we show that the resulting tests are often consistent for lower signal-to-noise ratios. Moreover, we establish an analogy between the power improvement and switching from a t-test to a Z-test under normality. Importantly, in permutation-based multiple testing, the efficiency gain with our approach can be huge, since we attain the same power with many fewer permutations.

Suggested Citation

  • N W Koning & J Hemerik, 2024. "More efficient exact group invariance testing: using a representative subgroup," Biometrika, Biometrika Trust, vol. 111(2), pages 441-458.
  • Handle: RePEc:oup:biomet:v:111:y:2024:i:2:p:441-458.
    as

    Download full text from publisher

    File URL: http://hdl.handle.net/10.1093/biomet/asad050
    Download Restriction: Access to full text is restricted to subscribers.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Jesse Hemerik & Jelle J. Goeman, 2021. "Another Look at the Lady Tasting Tea and Differences Between Permutation Tests and Randomisation Tests," International Statistical Review, International Statistical Institute, vol. 89(2), pages 367-381, August.
    2. Bekker, Paul A. & Lawford, Steve, 2008. "Symmetry-based inference in an instrumental variable setting," Journal of Econometrics, Elsevier, vol. 142(1), pages 28-49, January.
    3. Aaditya Ramdas & Rina Foygel Barber & Emmanuel J. Candès & Ryan J. Tibshirani, 2023. "Permutation Tests Using Arbitrary Permutation Distributions," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 85(2), pages 1156-1177, August.
    4. Jesse Hemerik & Jelle J. Goeman, 2018. "False discovery proportion estimation by permutations: confidence for significance analysis of microarrays," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 80(1), pages 137-155, January.
    5. Davidson, Russell & Flachaire, Emmanuel, 2008. "The wild bootstrap, tamed at last," Journal of Econometrics, Elsevier, vol. 146(1), pages 162-169, September.
    6. Jesse Hemerik & Jelle J. Goeman & Livio Finos, 2020. "Robust testing in generalized linear models by sign flipping score contributions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 841-864, July.
    7. Nicolai Meinshausen, 2006. "False Discovery Control for Multiple Tests of Association Under General Dependence," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 33(2), pages 227-237, June.
    8. Phipson Belinda & Smyth Gordon K, 2010. "Permutation P-values Should Never Be Zero: Calculating Exact P-values When Permutations Are Randomly Drawn," Statistical Applications in Genetics and Molecular Biology, De Gruyter, vol. 9(1), pages 1-16, October.
    9. Jesse Hemerik & Jelle Goeman, 2018. "Exact testing with random permutations," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(4), pages 811-825, December.
    10. Aldo Solari & Livio Finos & Jelle J. Goeman, 2014. "Rotation-based multiple testing in the multivariate linear model," Biometrics, The International Biometric Society, vol. 70(4), pages 954-961, December.
    11. Alwyn Young, 2019. "Channeling Fisher: Randomization Tests and the Statistical Insignificance of Seemingly Significant Experimental Results," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 134(2), pages 557-598.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jesse Hemerik & Jelle J. Goeman, 2021. "Another Look at the Lady Tasting Tea and Differences Between Permutation Tests and Randomisation Tests," International Statistical Review, International Statistical Institute, vol. 89(2), pages 367-381, August.
    2. David M. Ritzwoller & Joseph P. Romano & Azeem M. Shaikh, 2024. "Randomization Inference: Theory and Applications," Papers 2406.09521, arXiv.org.
    3. Ferraccioli, Federico & Sangalli, Laura M. & Finos, Livio, 2022. "Some first inferential tools for spatial regression with differential regularization," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    4. Jesse Hemerik & Jelle J. Goeman & Livio Finos, 2020. "Robust testing in generalized linear models by sign flipping score contributions," Journal of the Royal Statistical Society Series B, Royal Statistical Society, vol. 82(3), pages 841-864, July.
    5. Jesse Hemerik & Jelle Goeman, 2018. "Exact testing with random permutations," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 27(4), pages 811-825, December.
    6. Angel G. Angelov & Magnus Ekström, 2023. "Tests of stochastic dominance with repeated measurements data," AStA Advances in Statistical Analysis, Springer;German Statistical Society, vol. 107(3), pages 443-467, September.
    7. Guillermo Durand & Gilles Blanchard & Pierre Neuvial & Etienne Roquain, 2020. "Post hoc false positive control for structured hypotheses," Scandinavian Journal of Statistics, Danish Society for Theoretical Statistics;Finnish Statistical Society;Norwegian Statistical Association;Swedish Statistical Association, vol. 47(4), pages 1114-1148, December.
    8. MacKinnon, James G. & Nielsen, Morten Ørregaard & Webb, Matthew D., 2023. "Cluster-robust inference: A guide to empirical practice," Journal of Econometrics, Elsevier, vol. 232(2), pages 272-299.
    9. Young, Alwyn, 2024. "Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression," LSE Research Online Documents on Economics 120933, London School of Economics and Political Science, LSE Library.
    10. Aaditya Ramdas & Rina Foygel Barber & Emmanuel J. Candès & Ryan J. Tibshirani, 2023. "Permutation Tests Using Arbitrary Permutation Distributions," Sankhya A: The Indian Journal of Statistics, Springer;Indian Statistical Institute, vol. 85(2), pages 1156-1177, August.
    11. Young, Alwyn, 2024. "Asymptotically robust permutation-based randomization confidence intervals for parametric OLS regression," European Economic Review, Elsevier, vol. 163(C).
    12. Aksoy, Cevat Giray & Poutvaara, Panu & Schikora, Felicitas, 2023. "First time around: Local conditions and multi-dimensional integration of refugees," Journal of Urban Economics, Elsevier, vol. 137(C).
    13. Eichengreen, Barry & Aksoy, Cevat Giray & Saka, Orkun, 2021. "Revenge of the experts: Will COVID-19 renew or diminish public trust in science?," Journal of Public Economics, Elsevier, vol. 193(C).
    14. Goncalves, Silvia & Kilian, Lutz, 2004. "Bootstrapping autoregressions with conditional heteroskedasticity of unknown form," Journal of Econometrics, Elsevier, vol. 123(1), pages 89-120, November.
    15. Daiki Maki, 2015. "Wild bootstrap tests for unit root in ESTAR models," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 24(3), pages 475-490, September.
    16. Friedrich, Sarah & Pauly, Markus, 2018. "MATS: Inference for potentially singular and heteroscedastic MANOVA," Journal of Multivariate Analysis, Elsevier, vol. 165(C), pages 166-179.
    17. Andreoni, James & Serra-Garcia, Marta, 2021. "Time inconsistent charitable giving," Journal of Public Economics, Elsevier, vol. 198(C).
    18. Johnsen, Åshild A. & Kvaløy, Ola, 2021. "Conspiracy against the public - An experiment on collusion11“People of the same trade seldom meet together, even for merriment and diversion, but the conversation ends in a conspiracy against the publ," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 94(C).
    19. James G. MacKinnon & Matthew D. Webb & Morten Ø. Nielsen, 2017. "Bootstrap And Asymptotic Inference With Multiway Clustering," Working Paper 1386, Economics Department, Queen's University.
    20. Gonzalez, Felipe & Prem, Mounu, 2020. "Police Repression and Protest Behavior: Evidence from Student Protests in Chile," SocArXiv 3xk5r, Center for Open Science.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:oup:biomet:v:111:y:2024:i:2:p:441-458.. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Oxford University Press (email available below). General contact details of provider: https://academic.oup.com/biomet .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.