IDEAS home Printed from https://ideas.repec.org/a/bpj/ijbist/v14y2018i2p10n7.html
   My bibliography  Save this article

A Serious Flaw in Nutrition Epidemiology: A Meta-Analysis Study

Author

Listed:
  • Peace Karl E.

    (Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA)

  • Yin JingJing

    (Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA)

  • Rochani Haresh

    (Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA)

  • Pandeya Sarbesh

    (Jiann Ping-Hsu College of Public Health, Georgia Southern University, Statesboro, GA 30458, USA)

  • Young Stanley

    (CGSTAT, Raleigh, NC 27607, USA)

Abstract

BackgroundMany researchers have studied the relationship between diet and health. Specifically, there are papers showing an association between the consumption of sugar sweetened beverages and Type 2 diabetes. Many meta-analyses use individual studies that do not attempt to adjust for multiple testing or multiple modeling. Hence the claims reported in a meta-analysis paper may be unreliable as the base papers do not ensure unbiased statistics.ObjectiveDetermine (i) the statistical reliability of 10 papers and (ii) indirectly the reliability of the meta-analysis study.MethodWe obtained copies of each of the 10 papers used in a metaanalysis paper and counted the numbers of outcomes, predictors, and covariates. We estimate the size of the potential analysis search space available to the authors of these papers; i. e. the number of comparisons and models available. The potential analysis search space is the number of outcomes times the number of predictors times 2c, where c is the number of covariates. This formula was applied to information found in the abstracts (Space A) as well as the text (Space T) of each base paper.ResultsThe median and range of the number of comparisons possible across the base papers are 6.5 and (2 12,288), respectively for Space A, and 196,608 and (3072–117,117,952), respectively for Space T. It is noted that the median of 6.5 for Space A may be misleading as each study has 60–165 foods that could be predictors.ConclusionGiven that testing is at the 5% level and the number of comparisons is very large, nominal statistical significance is very weak support for a claim. The claims in these papers are not statistically supported and hence are unreliable so the meta-analysis paper is also unreliable.

Suggested Citation

  • Peace Karl E. & Yin JingJing & Rochani Haresh & Pandeya Sarbesh & Young Stanley, 2018. "A Serious Flaw in Nutrition Epidemiology: A Meta-Analysis Study," The International Journal of Biostatistics, De Gruyter, vol. 14(2), pages 1-10, November.
    • Handle: RePEc:bpj:ijbist:v:14:y:2018:i:2:p:10:n:7
    DOI: 10.1515/ijb-2018-0079
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/ijb-2018-0079
    Download Restriction: For access to full text, subscription to the journal or payment for the individual article is required.
    File URL: https://libkey.io/10.1515/ijb-2018-0079?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Edward L. Glaeser, 2006. "Researcher Incentives and Empirical Methods," NBER Technical Working Papers 0329, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Marcel Fafchamps & Julien Labonne, 2017. "Do Politicians’ Relatives Get Better Jobs? Evidence from Municipal Elections," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 33(2), pages 268-300.
    2. Alexander Frankel & Maximilian Kasy, 2022. "Which Findings Should Be Published?," American Economic Journal: Microeconomics, American Economic Association, vol. 14(1), pages 1-38, February.
    3. Robert L. Obenchain & S. Stanley Young, 2017. "Local Control Strategy: Simple Analyses of Air Pollution Data Can Reveal Heterogeneity in Longevity Outcomes," Risk Analysis, John Wiley & Sons, vol. 37(9), pages 1742-1753, September.
    4. Maximilian Kasy & Jann Spiess, 2022. "Optimal Pre-Analysis Plans: Statistical Decisions Subject to Implementability," Papers 2208.09638, arXiv.org, revised Jul 2024.
    5. Haeussler, Carolin, 2011. "Information-sharing in academia and the industry: A comparative study," Research Policy, Elsevier, vol. 40(1), pages 105-122, February.
    6. Necker, Sarah, 2014. "Scientific misbehavior in economics," Research Policy, Elsevier, vol. 43(10), pages 1747-1759.
    7. Maximilian Kasy & Jann Spiess, 2022. "Rationalizing Pre-Analysis Plans:Statistical Decisions Subject to Implementability," Economics Series Working Papers 975, University of Oxford, Department of Economics.
    8. Stephan B. Bruns, 2016. "The Fragility of Meta-Regression Models in Observational Research," MAGKS Papers on Economics 201603, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    9. Nicola Lacetera & Lorenzo Zirulia, 2011. "The Economics of Scientific Misconduct," The Journal of Law, Economics, and Organization, Oxford University Press, vol. 27(3), pages 568-603.
    10. Stephan B. Bruns & David I. Stern, 2015. "Meta-Granger causality testing," CAMA Working Papers 2015-22, Centre for Applied Macroeconomic Analysis, Crawford School of Public Policy, The Australian National University.
    11. Simonetta LONGHI & Peter NIJKAMP & Jacques POOT, 2008. "Meta-Analysis Of Empirical Evidence On The Labour Market Impacts Of Immigration," Region et Developpement, Region et Developpement, LEAD, Universite du Sud - Toulon Var, vol. 27, pages 161-191.
    12. Tomas Havranek & Anna Sokolova, 2016. "Do Consumers Really Follow a Rule of Thumb? Three Thousand Estimates from 130 Studies Say "Probably Not"," Working Papers 2016/08, Czech National Bank.
    13. Maximilian Kasy & Jann Spiess, 2024. "Optimal Pre-Analysis Plans: Statistical Decisions Subject to Implementability," Economics Series Working Papers 1058, University of Oxford, Department of Economics.
    14. Furman, Jeffrey L. & Jensen, Kyle & Murray, Fiona, 2012. "Governing knowledge in the scientific community: Exploring the role of retractions in biomedicine," Research Policy, Elsevier, vol. 41(2), pages 276-290.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:bpj:ijbist:v:14:y:2018:i:2:p:10:n:7. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.degruyter.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.