IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i7p1881-1893.html
   My bibliography  Save this article

An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA

Author

Listed:
  • Kherad-Pajouh, Sara
  • Renaud, Olivier

Abstract

The ANOVA method and permutation tests, two heritages of Fisher, have been extensively studied. Several permutation strategies have been proposed by others to obtain a distribution-free test for factors in a fixed effect ANOVA (i.e., single error term ANOVA). The resulting tests are either approximate or exact. However, there exists no universal exact permutation test which can be applied to an arbitrary design to test a desired factor. An exact permutation strategy applicable to fixed effect analysis of variance is presented. The proposed method can be used to test any factor, even in the presence of higher-order interactions. In addition, the method has the advantage of being applicable in unbalanced designs (all-cell-filled), which is a very common situation in practice, and it is the first method with this capability. Simulation studies show that the proposed method has an actual level which stays remarkably close to the nominal level, and its power is always competitive. This is the case even with very small datasets, strongly unbalanced designs and non-Gaussian errors. No other competitor show such an enviable behavior.

Suggested Citation

  • Kherad-Pajouh, Sara & Renaud, Olivier, 2010. "An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1881-1893, July.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(10)00080-0
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. David, Herbert A., 2008. "The Beginnings of Randomization Tests," The American Statistician, American Statistical Association, vol. 62, pages 70-72, February.
    2. Brombin, Chiara & Salmaso, Luigi, 2009. "Multi-aspect permutation tests in shape analysis with small sample size," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 3921-3931, October.
    3. Freedman, David & Lane, David, 1983. "A Nonstochastic Interpretation of Reported Significance Levels," Journal of Business & Economic Statistics, American Statistical Association, vol. 1(4), pages 292-298, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sonja Hahn & Luigi Salmaso, 2017. "A comparison of different synchronized permutation approaches to testing effects in two-level two-factor unbalanced ANOVA designs," Statistical Papers, Springer, vol. 58(1), pages 123-146, March.
    2. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    3. Finos, Livio, 2011. "A note on Left-Spherically Distributed test with covariates," Statistics & Probability Letters, Elsevier, vol. 81(6), pages 639-641, June.
    4. Ferraccioli, Federico & Sangalli, Laura M. & Finos, Livio, 2022. "Some first inferential tools for spatial regression with differential regularization," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    5. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Niels Lundtorp Olsen & Alessia Pini & Simone Vantini, 2021. "False discovery rate for functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 784-809, September.
    2. Stephen Klein & David Freedman & Richard Shavelson & Roger Bolus, 2008. "Assessing School Effectiveness," Evaluation Review, , vol. 32(6), pages 511-525, December.
    3. Orla Doyle & Liam Delaney & Christine O'Farrelly & Nick Fitzpatrick & Michael Daly, 2015. "Can Early Intervention Improve Maternal Well-being? Evidence from a Randomized Controlled Trial," Working Papers 2015-015, Human Capital and Economic Opportunity Working Group.
    4. Stefano Bonnini & Michela Borghesi, 2022. "Relationship between Mental Health and Socio-Economic, Demographic and Environmental Factors in the COVID-19 Lockdown Period—A Multivariate Regression Analysis," Mathematics, MDPI, vol. 10(18), pages 1-15, September.
    5. repec:jss:jstsof:07:i10 is not listed on IDEAS
    6. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jul 2024.
    7. Orla Doyle, 2017. "The First 2,000 Days and Child Skills: Evidence from a Randomized Experiment of Home Visiting," Working Papers 2017-054, Human Capital and Economic Opportunity Working Group.
    8. Charles A. Holt & Sean P. Sullivan, 2023. "Permutation tests for experimental data," Experimental Economics, Springer;Economic Science Association, vol. 26(4), pages 775-812, September.
    9. Chiara Brombin & Luigi Salmaso & Lara Fontanella & Luigi Ippoliti, 2015. "Nonparametric combination-based tests in dynamic shape analysis," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 27(4), pages 460-484, December.
    10. Juliana Menger & William E Magnusson & Marti J Anderson & Martin Schlegel & Guy Pe’er & Klaus Henle, 2017. "Environmental characteristics drive variation in Amazonian understorey bird assemblages," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-20, February.
    11. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    12. Hu, Yuan & Behrman, Jere R. & Zhang, Junsen, 2021. "The causal effects of parents’ schooling on children's schooling in urban China," Journal of Comparative Economics, Elsevier, vol. 49(1), pages 258-276.
    13. Orla Doyle, 2024. "Can Early Intervention Have a Sustained Effect on Human Capital?," Journal of Human Resources, University of Wisconsin Press, vol. 59(5), pages 1599-1636.
    14. Michael Daly & Liam Delaney & Orla Doyle & Nick Fitzpatrick & Christine O'Farrelly, 2014. "Can Early Intervention Policies Improve Well-being? Evidence from a randomized controlled trial," Working Papers 201415, School of Economics, University College Dublin.
    15. Gill,Indermit S., 1990. "Does the structure of production affect demand for schooling in Peru?," Policy Research Working Paper Series 468, The World Bank.
    16. Agostino Torti & Alessia Pini & Simone Vantini, 2021. "Modelling time‐varying mobility flows using function‐on‐function regression: Analysis of a bike sharing system in the city of Milan," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 70(1), pages 226-247, January.
    17. Bonnet, Eric & de Peer, Yves Van, 2002. "zt: A Sofware Tool for Simple and Partial Mantel Tests," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 7(i10).
    18. Pinto, Rodrigo, 2010. "Evaluation of Small-sample Compromised Randomization: Long-term Effects of Early Childhood Intervention on Health and Addictive Behavior," Brazilian Review of Econometrics, Sociedade Brasileira de Econometria - SBE, vol. 30(2), December.
    19. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    20. Pini, Alessia & Stamm, Aymeric & Vantini, Simone, 2018. "Hotelling’s T2 in separable Hilbert spaces," Journal of Multivariate Analysis, Elsevier, vol. 167(C), pages 284-305.
    21. Mittelhammer, Ron C. & Judge, George G., 2005. "Combining estimators to improve structural model estimation and inference under quadratic loss," Journal of Econometrics, Elsevier, vol. 128(1), pages 1-29, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.