IDEAS home Printed from https://ideas.repec.org/a/bla/biomet/v74y2018i1p362-368.html
   My bibliography  Save this article

Why you cannot transform your way out of trouble for small counts

Author

Listed:
  • David I. Warton

Abstract

While data transformation is a common strategy to satisfy linear modeling assumptions, a theoretical result is used to show that transformation cannot reasonably be expected to stabilize variances for small counts. Under broad assumptions, as counts get smaller, it is shown that the variance becomes proportional to the mean under monotonic transformations g(·) that satisfy g(0)=0, excepting a few pathological cases. A suggested rule†of†thumb is that if many predicted counts are less than one then data transformation cannot reasonably be expected to stabilize variances, even for a well†chosen transformation. This result has clear implications for the analysis of counts as often implemented in the applied sciences, but particularly for multivariate analysis in ecology. Multivariate discrete data are often collected in ecology, typically with a large proportion of zeros, and it is currently widespread to use methods of analysis that do not account for differences in variance across observations nor across responses. Simulations demonstrate that failure to account for the mean–variance relationship can have particularly severe consequences in this context, and also in the univariate context if the sampling design is unbalanced.

Suggested Citation

  • David I. Warton, 2018. "Why you cannot transform your way out of trouble for small counts," Biometrics, The International Biometric Society, vol. 74(1), pages 362-368, March.
  • Handle: RePEc:bla:biomet:v:74:y:2018:i:1:p:362-368
    DOI: 10.1111/biom.12728
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/biom.12728
    Download Restriction: no

    File URL: https://libkey.io/10.1111/biom.12728?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Cameron,A. Colin & Trivedi,Pravin K., 2013. "Regression Analysis of Count Data," Cambridge Books, Cambridge University Press, number 9781107667273, September.
    2. Philip T. Reiss & M. Henry H. Stevens & Zarrar Shehzad & Eva Petkova & Michael P. Milham, 2010. "On Distance-Based Permutation Tests for Between-Group Comparisons," Biometrics, The International Biometric Society, vol. 66(2), pages 636-643, June.
    3. Irène Gijbels & Marek Omelka, 2013. "Testing for Homogeneity of Multivariate Dispersions Using Dissimilarity Measures," Biometrics, The International Biometric Society, vol. 69(1), pages 137-145, March.
    4. Marti J. Anderson, 2006. "Distance-Based Tests for Homogeneity of Multivariate Dispersions," Biometrics, The International Biometric Society, vol. 62(1), pages 245-253, March.
    5. Jun Li & Jifei Ban & Louis S. Santiago, 2011. "Nonparametric Tests for Homogeneity of Species Assemblages: A Data Depth Approach," Biometrics, The International Biometric Society, vol. 67(4), pages 1481-1488, December.
    6. David I. Warton, 2011. "Regularized Sandwich Estimators for Analysis of High-Dimensional Data Using Generalized Estimating Equations," Biometrics, The International Biometric Society, vol. 67(1), pages 116-123, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Alejandra Goldenberg Vilar & Timme Donders & Aleksandra Cvetkoska & Friederike Wagner-Cremer, 2018. "Seasonality modulates the predictive skills of diatom based salinity transfer functions," PLOS ONE, Public Library of Science, vol. 13(11), pages 1-19, November.
    2. Wang, Xu & Zhang, Xiaobo & Xie, Zhuan & Huang, Yiping, 2016. "Roads to innovation: Firm-level evidence from China:," IFPRI discussion papers 1542, International Food Policy Research Institute (IFPRI).
    3. Preusse, Verena & Wollni, Meike, 2021. "Adoption of sustainable agricultural practices in the context of urbanisation and environmental stress – Evidence from farmers in the rural-urban interface of Bangalore, India," 2021 Annual Meeting, August 1-3, Austin, Texas 312690, Agricultural and Applied Economics Association.
    4. Luiz Paulo Fávero & Joseph F. Hair & Rafael de Freitas Souza & Matheus Albergaria & Talles V. Brugni, 2021. "Zero-Inflated Generalized Linear Mixed Models: A Better Way to Understand Data Relationships," Mathematics, MDPI, vol. 9(10), pages 1-28, May.
    5. Bono, Pierre-Henri & David, Quentin & Desbordes, Rodolphe & Py, Loriane, 2022. "Metro infrastructure and metropolitan attractiveness," Regional Science and Urban Economics, Elsevier, vol. 93(C).
    6. Scott, Ryan P. & Scott, Tyler A., 2019. "Investing in collaboration for safety: Assessing grants to states for oil and gas distribution pipeline safety program enhancement," Energy Policy, Elsevier, vol. 124(C), pages 332-345.
    7. Riccardo (Jack) Lucchetti & Luca Pedini, 2020. "ParMA: Parallelised Bayesian Model Averaging for Generalised Linear Models," Working Papers 2020:28, Department of Economics, University of Venice "Ca' Foscari".
    8. Landry, Craig E. & Shonkwiler, J. Scott & Whitehead, John C., 2020. "Economic Values of Coastal Erosion Management: Joint Estimation of Use and Existence Values with recreation demand and contingent valuation data," Journal of Environmental Economics and Management, Elsevier, vol. 103(C).
    9. John McLaren & Su Wang, 2020. "Effects of Reduced Workplace Presence on COVID-19 Deaths: An Instrumental-Variables Approach," NBER Working Papers 28275, National Bureau of Economic Research, Inc.
    10. Massimiliano Cal� & Sami H. Miaari, 2014. "Trade, employment and conflict: Evidence from the Second Intifada," HiCN Working Papers 186, Households in Conflict Network.
    11. Kauffmann, Albrecht, 2021. "Befindet sich die "Metropolregion Mitteldeutschland" auf dem Weg zur räumlich integrierten Region? Eine empirische Untersuchung der Berufspendlerverflechtungen," Arbeitsberichte der ARL: Aufsätze, in: Rosenfeld, Martin T. W. & Stefansky, Andreas (ed.), "Metropolregion Mitteldeutschland" aus raumwissenschaftlicher Sicht, volume 30, pages 76-95, ARL – Akademie für Raumentwicklung in der Leibniz-Gemeinschaft.
    12. Barfield, Ashley & Shonkwiler, J. Scott, 2016. "A Distribution Transition Method for Extreme Responses in Recreation Survey Data," 2016 Annual Meeting, July 31-August 2, Boston, Massachusetts 235670, Agricultural and Applied Economics Association.
    13. Ghosh, Prasenjit & Rong, Jian & Khanna, Madhu & Wang, Weiwei & Miao, Ruiqing, 2017. "Have They Gone with the Wind? Indirect Effects of Wind Turbines on Bird Abundance," 2017 Annual Meeting, July 30-August 1, Chicago, Illinois 258100, Agricultural and Applied Economics Association.
    14. Faia, Ester & Ottaviano, Gianmarco & Sanchez Arjona, Irene, 2017. "International Expansion and Riskiness of Banks," CEPR Discussion Papers 11951, C.E.P.R. Discussion Papers.
    15. Mullahy, John, 2024. "Analyzing health outcomes measured as bounded counts," Journal of Health Economics, Elsevier, vol. 95(C).
    16. Michel Beine & Ilan Noy & Christopher Parsons, 2021. "Climate change, migration and voice," Climatic Change, Springer, vol. 167(1), pages 1-27, July.
    17. Meng Xu & Philip T. Reiss & Ivor Cribben, 2021. "Generalized reliability based on distances," Biometrics, The International Biometric Society, vol. 77(1), pages 258-270, March.
    18. Christian Kleiber & Achim Zeileis, 2016. "Visualizing Count Data Regressions Using Rootograms," The American Statistician, Taylor & Francis Journals, vol. 70(3), pages 296-303, July.
    19. D M Zimmer, 2023. "The effect of food stamps on fibre intake," Economic Issues Journal Articles, Economic Issues, vol. 28(2), pages 71-86, September.
    20. Cirillo, Valeria & Fanti, Lucrezia & Mina, Andrea & Ricci, Andrea, 2023. "The adoption of digital technologies: Investment, skills, work organisation," Structural Change and Economic Dynamics, Elsevier, vol. 66(C), pages 89-105.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bla:biomet:v:74:y:2018:i:1:p:362-368. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: http://www.blackwellpublishing.com/journal.asp?ref=0006-341X .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.