IDEAS home Printed from https://ideas.repec.org/a/eee/csdana/v54y2010i7p1881-1893.html
   My bibliography  Save this article

An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA

Author

Listed:
  • Kherad-Pajouh, Sara
  • Renaud, Olivier

Abstract

The ANOVA method and permutation tests, two heritages of Fisher, have been extensively studied. Several permutation strategies have been proposed by others to obtain a distribution-free test for factors in a fixed effect ANOVA (i.e., single error term ANOVA). The resulting tests are either approximate or exact. However, there exists no universal exact permutation test which can be applied to an arbitrary design to test a desired factor. An exact permutation strategy applicable to fixed effect analysis of variance is presented. The proposed method can be used to test any factor, even in the presence of higher-order interactions. In addition, the method has the advantage of being applicable in unbalanced designs (all-cell-filled), which is a very common situation in practice, and it is the first method with this capability. Simulation studies show that the proposed method has an actual level which stays remarkably close to the nominal level, and its power is always competitive. This is the case even with very small datasets, strongly unbalanced designs and non-Gaussian errors. No other competitor show such an enviable behavior.

Suggested Citation

  • Kherad-Pajouh, Sara & Renaud, Olivier, 2010. "An exact permutation method for testing any effect in balanced and unbalanced fixed effect ANOVA," Computational Statistics & Data Analysis, Elsevier, vol. 54(7), pages 1881-1893, July.
    • Handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0167-9473(10)00080-0
    Download Restriction: Full text for ScienceDirect subscribers only.
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. David, Herbert A., 2008. "The Beginnings of Randomization Tests," The American Statistician, American Statistical Association, vol. 62, pages 70-72, February.
    2. Brombin, Chiara & Salmaso, Luigi, 2009. "Multi-aspect permutation tests in shape analysis with small sample size," Computational Statistics & Data Analysis, Elsevier, vol. 53(12), pages 3921-3931, October.
    3. Freedman, David & Lane, David, 1983. "A Nonstochastic Interpretation of Reported Significance Levels," Journal of Business & Economic Statistics, American Statistical Association, vol. 1(4), pages 292-298, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sara Kherad-Pajouh & Olivier Renaud, 2015. "A general permutation approach for analyzing repeated measures ANOVA and mixed-model designs," Statistical Papers, Springer, vol. 56(4), pages 947-967, November.
    2. Ferraccioli, Federico & Sangalli, Laura M. & Finos, Livio, 2022. "Some first inferential tools for spatial regression with differential regularization," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    3. Finos, Livio, 2011. "A note on Left-Spherically Distributed test with covariates," Statistics & Probability Letters, Elsevier, vol. 81(6), pages 639-641, June.
    4. Sonja Hahn & Luigi Salmaso, 2017. "A comparison of different synchronized permutation approaches to testing effects in two-level two-factor unbalanced ANOVA designs," Statistical Papers, Springer, vol. 58(1), pages 123-146, March.
    5. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Daly, Michael & Delaney, Liam & Doyle, Orla & Fitzpatrick, Nick & O'Farrelly, Christine, 2014. "Can Early Intervention Policies Improve Well-being? Evidence from a randomized controlled trial," 2007 Annual Meeting, July 29-August 1, 2007, Portland, Oregon TN 2015-03, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    2. Niels Lundtorp Olsen & Alessia Pini & Simone Vantini, 2021. "False discovery rate for functional data," TEST: An Official Journal of the Spanish Society of Statistics and Operations Research, Springer;Sociedad de Estadística e Investigación Operativa, vol. 30(3), pages 784-809, September.
    3. Římalová, Veronika & Fišerová, Eva & Menafoglio, Alessandra & Pini, Alessia, 2022. "Inference for spatial regression models with functional response using a permutational approach," Journal of Multivariate Analysis, Elsevier, vol. 189(C).
    4. Stephen Klein & David Freedman & Richard Shavelson & Roger Bolus, 2008. "Assessing School Effectiveness," Evaluation Review, , vol. 32(6), pages 511-525, December.
    5. Karel Hron & Jitka Machalová & Alessandra Menafoglio, 2023. "Bivariate densities in Bayes spaces: orthogonal decomposition and spline representation," Statistical Papers, Springer, vol. 64(5), pages 1629-1667, October.
    6. Soo Hong Chew & Junjian Yi & Junsen Zhang & Songfa Zhong, 2016. "Education and anomalies in decision making: Experimental evidence from Chinese adult twins," Journal of Risk and Uncertainty, Springer, vol. 53(2), pages 163-200, December.
    7. Orla Doyle & Liam Delaney & Christine O'Farrelly & Nick Fitzpatrick & Michael Daly, 2015. "Can Early Intervention Improve Maternal Well-being? Evidence from a Randomized Controlled Trial," Working Papers 2015-015, Human Capital and Economic Opportunity Working Group.
    8. Tomáš Mrkvička & Tomáš Roskovec & Michael Rost, 2021. "A Nonparametric Graphical Tests of Significance in Functional GLM," Methodology and Computing in Applied Probability, Springer, vol. 23(2), pages 593-612, June.
    9. James J. Heckman & Margaret L. Holland & Kevin K. Makino & Rodrigo Pinto & Maria Rosales-Rueda, 2017. "An Analysis of the Memphis Nurse-Family Partnership Program," NBER Working Papers 23610, National Bureau of Economic Research, Inc.
    10. Stefano Bonnini & Michela Borghesi, 2022. "Relationship between Mental Health and Socio-Economic, Demographic and Environmental Factors in the COVID-19 Lockdown Period—A Multivariate Regression Analysis," Mathematics, MDPI, vol. 10(18), pages 1-15, September.
    11. repec:jss:jstsof:07:i10 is not listed on IDEAS
    12. Purevdorj Tuvaandorj, 2021. "Robust Permutation Tests in Linear Instrumental Variables Regression," Papers 2111.13774, arXiv.org, revised Jul 2024.
    13. Bryan Keller, 2012. "Detecting Treatment Effects with Small Samples: The Power of Some Tests Under the Randomization Model," Psychometrika, Springer;The Psychometric Society, vol. 77(2), pages 324-338, April.
    14. Orla Doyle, 2017. "The First 2,000 Days and Child Skills: Evidence from a Randomized Experiment of Home Visiting," Working Papers 2017-054, Human Capital and Economic Opportunity Working Group.
    15. David Dekker & David Krackhardt & Tom Snijders, 2007. "Sensitivity of MRQAP Tests to Collinearity and Autocorrelation Conditions," Psychometrika, Springer;The Psychometric Society, vol. 72(4), pages 563-581, December.
    16. Charles A. Holt & Sean P. Sullivan, 2023. "Permutation tests for experimental data," Experimental Economics, Springer;Economic Science Association, vol. 26(4), pages 775-812, September.
    17. Veronika Římalová & Alessandra Menafoglio & Alessia Pini & Vilém Pechanec & Eva Fišerová, 2020. "A permutation approach to the analysis of spatiotemporal geochemical data in the presence of heteroscedasticity," Environmetrics, John Wiley & Sons, Ltd., vol. 31(4), June.
    18. Chiara Brombin & Luigi Salmaso & Lara Fontanella & Luigi Ippoliti, 2015. "Nonparametric combination-based tests in dynamic shape analysis," Journal of Nonparametric Statistics, Taylor & Francis Journals, vol. 27(4), pages 460-484, December.
    19. Juliana Menger & William E Magnusson & Marti J Anderson & Martin Schlegel & Guy Pe’er & Klaus Henle, 2017. "Environmental characteristics drive variation in Amazonian understorey bird assemblages," PLOS ONE, Public Library of Science, vol. 12(2), pages 1-20, February.
    20. Stefano Bonnini & Getnet Melak Assegie & Kamila Trzcinska, 2024. "Review about the Permutation Approach in Hypothesis Testing," Mathematics, MDPI, vol. 12(17), pages 1-29, August.
    21. Daly, Michael & Delaney, Liam & Doyle, Orla & Fitzpatrick, Nick & O'Farrelly, Christine, 2014. "Can Early Intervention Policies Improve Well-being? Evidence from a randomized controlled trial," SIRE Discussion Papers 2015-03, Scottish Institute for Research in Economics (SIRE).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:csdana:v:54:y:2010:i:7:p:1881-1893. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/csda .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.