The Ultimate Cause of the “Reproducibility Crisis”: Reductionist Statistics
Author
Abstract
Suggested Citation
DOI: 10.31219/osf.io/yxba5
Download full text from publisher
References listed on IDEAS
- Alessandro Selvitella, 2017. "The ubiquity of the Simpson’s Paradox," Journal of Statistical Distributions and Applications, Springer, vol. 4(1), pages 1-16, December.
- David Colquhoun, 2019. "The False Positive Risk: A Proposal Concerning What to Do About p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 192-201, March.
- Valentin Amrhein & David Trafimow & Sander Greenland, 2019. "Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 262-270, March.
- Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
- Blakeley B. McShane & Jennifer L. Tackett & Ulf Böckenholt & Andrew Gelman, 2019. "Large-Scale Replication Projects in Contemporary Psychological Research," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 99-105, March.
- Leonid Hanin, 2021. "Cavalier Use of Inferential Statistics Is a Major Source of False and Irreproducible Scientific Findings," Mathematics, MDPI, vol. 9(6), pages 1-13, March.
- Takuji Usui & Malcolm R Macleod & Sarah K McCann & Alistair M Senior & Shinichi Nakagawa, 2021. "Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research," PLOS Biology, Public Library of Science, vol. 19(5), pages 1-20, May.
- Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521643368, January.
- Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018.
"Redefine statistical significance,"
Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
- Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
- Raymond Hubbard & Brian D. Haig & Rahul A. Parsa, 2019. "The Limited Role of Formal Statistical Inference in Scientific Inference," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 91-98, March.
- Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521644112, January.
- Scott Marek & Brenden Tervo-Clemmens & Finnegan J. Calabro & David F. Montez & Benjamin P. Kay & Alexander S. Hatoum & Meghan Rose Donohue & William Foran & Ryland L. Miller & Timothy J. Hendrickson &, 2022. "Reproducible brain-wide association studies require thousands of individuals," Nature, Nature, vol. 603(7902), pages 654-660, March.
- Shareen A Iqbal & Joshua D Wallach & Muin J Khoury & Sheri D Schully & John P A Ioannidis, 2016. "Reproducible Research Practices and Transparency across the Biomedical Literature," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-13, January.
- Blakeley B. McShane & David Gal, 2017. "Rejoinder: Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 904-908, July.
- Nicole C Nelson & Kelsey Ichikawa & Julie Chung & Momin M Malik, 2021. "Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis," PLOS ONE, Public Library of Science, vol. 16(7), pages 1-20, July.
- Leonard P Freedman & Iain M Cockburn & Timothy S Simcoe, 2015. "The Economics of Reproducibility in Preclinical Research," PLOS Biology, Public Library of Science, vol. 13(6), pages 1-9, June.
- Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
- Matthew J Page & Larissa Shamseer & Douglas G Altman & Jennifer Tetzlaff & Margaret Sampson & Andrea C Tricco & Ferrán Catalá-López & Lun Li & Emma K Reid & Rafael Sarkis-Onofre & David Moher, 2016. "Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study," PLOS Medicine, Public Library of Science, vol. 13(5), pages 1-30, May.
- Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
- Blakeley B. McShane & David Gal, 2017. "Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 885-895, July.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Sadri, Arash, 2022. "Machine Learning Can Solve the Reproducibility Crisis by Supplanting Reductionist Statistics," MetaArXiv yxba5_v1, Center for Open Science.
- Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018.
"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,"
Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6_v1, Center for Open Science.
- Camerer, Colin F. & Dreber, Anna & Holzmeister, Felix & Ho, Teck-Hua & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas & Altmejd, Adam & But, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Munich Reprints in Economics 62818, University of Munich, Department of Economics.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6, Center for Open Science.
- Christopher Allen & David M A Mehler, 2019. "Open science challenges, benefits and tips in early career and beyond," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-14, May.
- Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
- Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023.
"Heterogeneity in effect size estimates: Empirical evidence and practical implications,"
Working Papers
2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
- Holzmeister, Felix & Johannesson, Magnus & Böhm, Robert & Dreber, Anna & Huber, Jürgen & Kirchler, Michael, 2024. "Heterogeneity in Effect Size Estimates: Empirical Evidence and Practical Implications," I4R Discussion Paper Series 102, The Institute for Replication (I4R).
- Luigi Pace & Alessandra Salvan, 2020. "Likelihood, Replicability and Robbins' Confidence Sequences," International Statistical Review, International Statistical Institute, vol. 88(3), pages 599-615, December.
- Glenn Shafer, 2021. "Testing by betting: A strategy for statistical and scientific communication," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 407-431, April.
- Strømland, Eirik, 2019. "Preregistration and reproducibility," Journal of Economic Psychology, Elsevier, vol. 75(PA).
- Tom Engsted, 2024. "What Is the False Discovery Rate in Empirical Research?," Econ Journal Watch, Econ Journal Watch, vol. 21(1), pages 1-92–112, March.
- Maier, Maximilian & VanderWeele, Tyler J. & Mathur, Maya B, 2021. "Using Selection Models to Assess Sensitivity to Publication Bias: A Tutorial and Call for More Routine Use," MetaArXiv tp45u_v1, Center for Open Science.
- Julian Reiss, 2001. "Natural economic quantities and their measurement," Journal of Economic Methodology, Taylor & Francis Journals, vol. 8(2), pages 287-311.
- Matthew Rosenblatt & Link Tejavibulya & Rongtao Jiang & Stephanie Noble & Dustin Scheinost, 2024. "Data leakage inflates prediction performance in connectome-based machine learning models," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
- Aumann, Craig A., 2007. "A methodology for developing simulation models of complex systems," Ecological Modelling, Elsevier, vol. 202(3), pages 385-396.
- Stephen Pratten, 2007. "Realism, closed systems and abstraction," Journal of Economic Methodology, Taylor & Francis Journals, vol. 14(4), pages 473-497.
- Bertoldi, Paolo & Mosconi, Rocco, 2020. "Do energy efficiency policies save energy? A new approach based on energy policy indicators (in the EU Member States)," Energy Policy, Elsevier, vol. 139(C).
- Midgley, Gerald, 2008. "Response to paper "Systems thinking" by D. Cabrera et al.:: The unification of systems thinking: Is there gold at the end of the rainbow?," Evaluation and Program Planning, Elsevier, vol. 31(3), pages 317-321, August.
- Nicolas Brisset, 2018.
"Models as speech acts: the telling case of financial models,"
Journal of Economic Methodology, Taylor & Francis Journals, vol. 25(1), pages 21-41, January.
- Nicolas Brisset, 2017. "Models as Speech Acts: The Telling Case of Financial Models," GREDEG Working Papers 2017-25, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
- Toby Ord & Rafaela Hillerbrand & Anders Sandberg, 2010. "Probing the improbable: methodological challenges for risks with low probabilities and high stakes," Journal of Risk Research, Taylor & Francis Journals, vol. 13(2), pages 191-205, March.
- Simon Hall & Nilufa Ali & Nick Chater & Mike Oaksford, 2016. "Discounting and Augmentation in Causal Conditional Reasoning: Causal Models or Shallow Encoding?," PLOS ONE, Public Library of Science, vol. 11(12), pages 1-23, December.
- David J. Hand, 2022. "Trustworthiness of statistical inference," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(1), pages 329-347, January.
More about this item
NEP fields
This paper has been announced in the following NEP Reports:- NEP-HME-2022-06-13 (Heterodox Microeconomics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:yxba5. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.