The Ultimate Cause of the “Reproducibility Crisis”: Reductionist Statistics
Author
Abstract
Suggested Citation
DOI: 10.31219/osf.io/yxba5
Download full text from publisher
References listed on IDEAS
- Daniel J. Benjamin & James O. Berger & Magnus Johannesson & Brian A. Nosek & E.-J. Wagenmakers & Richard Berk & Kenneth A. Bollen & Björn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Chr, 2018.
"Redefine statistical significance,"
Nature Human Behaviour, Nature, vol. 2(1), pages 6-10, January.
- Daniel Benjamin & James Berger & Magnus Johannesson & Brian Nosek & E. Wagenmakers & Richard Berk & Kenneth Bollen & Bjorn Brembs & Lawrence Brown & Colin Camerer & David Cesarini & Christopher Chambe, 2017. "Redefine Statistical Significance," Artefactual Field Experiments 00612, The Field Experiments Website.
- Alessandro Selvitella, 2017. "The ubiquity of the Simpson’s Paradox," Journal of Statistical Distributions and Applications, Springer, vol. 4(1), pages 1-16, December.
- David Colquhoun, 2019. "The False Positive Risk: A Proposal Concerning What to Do About p-Values," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 192-201, March.
- Valentin Amrhein & David Trafimow & Sander Greenland, 2019. "Inferential Statistics as Descriptive Statistics: There Is No Replication Crisis if We Don’t Expect Replication," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 262-270, March.
- Raymond Hubbard & Brian D. Haig & Rahul A. Parsa, 2019. "The Limited Role of Formal Statistical Inference in Scientific Inference," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 91-98, March.
- Bernhard Voelkl & Lucile Vogt & Emily S Sena & Hanno Würbel, 2018. "Reproducibility of preclinical animal research improves with heterogeneity of study samples," PLOS Biology, Public Library of Science, vol. 16(2), pages 1-13, February.
- Blakeley B. McShane & Jennifer L. Tackett & Ulf Böckenholt & Andrew Gelman, 2019. "Large-Scale Replication Projects in Contemporary Psychological Research," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 99-105, March.
- Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521644112, November.
- Scott Marek & Brenden Tervo-Clemmens & Finnegan J. Calabro & David F. Montez & Benjamin P. Kay & Alexander S. Hatoum & Meghan Rose Donohue & William Foran & Ryland L. Miller & Timothy J. Hendrickson &, 2022. "Reproducible brain-wide association studies require thousands of individuals," Nature, Nature, vol. 603(7902), pages 654-660, March.
- Leonid Hanin, 2021. "Cavalier Use of Inferential Statistics Is a Major Source of False and Irreproducible Scientific Findings," Mathematics, MDPI, vol. 9(6), pages 1-13, March.
- Shareen A Iqbal & Joshua D Wallach & Muin J Khoury & Sheri D Schully & John P A Ioannidis, 2016. "Reproducible Research Practices and Transparency across the Biomedical Literature," PLOS Biology, Public Library of Science, vol. 14(1), pages 1-13, January.
- Blakeley B. McShane & David Gal, 2017. "Rejoinder: Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 904-908, July.
- Nicole C Nelson & Kelsey Ichikawa & Julie Chung & Momin M Malik, 2021. "Mapping the discursive dimensions of the reproducibility crisis: A mixed methods analysis," PLOS ONE, Public Library of Science, vol. 16(7), pages 1-20, July.
- Leonard P Freedman & Iain M Cockburn & Timothy S Simcoe, 2015. "The Economics of Reproducibility in Preclinical Research," PLOS Biology, Public Library of Science, vol. 13(6), pages 1-9, June.
- Takuji Usui & Malcolm R Macleod & Sarah K McCann & Alistair M Senior & Shinichi Nakagawa, 2021. "Meta-analysis of variation suggests that embracing variability improves both replicability and generalizability in preclinical research," PLOS Biology, Public Library of Science, vol. 19(5), pages 1-20, May.
- Rotem Botvinik-Nezer & Felix Holzmeister & Colin F. Camerer & Anna Dreber & Juergen Huber & Magnus Johannesson & Michael Kirchler & Roni Iwanir & Jeanette A. Mumford & R. Alison Adcock & Paolo Avesani, 2020. "Variability in the analysis of a single neuroimaging dataset by many teams," Nature, Nature, vol. 582(7810), pages 84-88, June.
- Matthew J Page & Larissa Shamseer & Douglas G Altman & Jennifer Tetzlaff & Margaret Sampson & Andrea C Tricco & Ferrán Catalá-López & Lun Li & Emma K Reid & Rafael Sarkis-Onofre & David Moher, 2016. "Epidemiology and Reporting Characteristics of Systematic Reviews of Biomedical Research: A Cross-Sectional Study," PLOS Medicine, Public Library of Science, vol. 13(5), pages 1-30, May.
- Cartwright,Nancy, 1999. "The Dappled World," Cambridge Books, Cambridge University Press, number 9780521643368, November.
- Nosek, Brian A. & Ebersole, Charles R. & DeHaven, Alexander Carl & Mellor, David Thomas, 2018. "The Preregistration Revolution," OSF Preprints 2dxu5, Center for Open Science.
- Blakeley B. McShane & David Gal, 2017. "Statistical Significance and the Dichotomization of Evidence," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 112(519), pages 885-895, July.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Colin F. Camerer & Anna Dreber & Felix Holzmeister & Teck-Hua Ho & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Gideon Nave & Brian A. Nosek & Thomas Pfeiffer & Adam Altmejd & Nick Buttrick , 2018.
"Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015,"
Nature Human Behaviour, Nature, vol. 2(9), pages 637-644, September.
- Camerer, Colin & Dreber, Anna & Holzmeister, Felix & Ho, Teck Hua & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," SocArXiv 4hmb6, Center for Open Science.
- Camerer, Colin F. & Dreber, Anna & Holzmeister, Felix & Ho, Teck-Hua & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Nave, Gideon & Nosek, Brian A. & Pfeiffer, Thomas & Altmejd, Adam & But, 2018. "Evaluating the replicability of social science experiments in Nature and Science between 2010 and 2015," Munich Reprints in Economics 62818, University of Munich, Department of Economics.
- Christopher Allen & David M A Mehler, 2019. "Open science challenges, benefits and tips in early career and beyond," PLOS Biology, Public Library of Science, vol. 17(5), pages 1-14, May.
- Tom Engsted, 2024. "What Is the False Discovery Rate in Empirical Research?," Econ Journal Watch, Econ Journal Watch, vol. 21(1), pages 1-92–112, March.
- Gunter, Ulrich & Önder, Irem & Smeral, Egon, 2019. "Scientific value of econometric tourism demand studies," Annals of Tourism Research, Elsevier, vol. 78(C), pages 1-1.
- Felix Holzmeister & Magnus Johannesson & Robert Böhm & Anna Dreber & Jürgen Huber & Michael Kirchler, 2023.
"Heterogeneity in effect size estimates: Empirical evidence and practical implications,"
Working Papers
2023-17, Faculty of Economics and Statistics, Universität Innsbruck.
- Holzmeister, Felix & Johannesson, Magnus & Böhm, Robert & Dreber, Anna & Huber, Jürgen & Kirchler, Michael, 2024. "Heterogeneity in Effect Size Estimates: Empirical Evidence and Practical Implications," I4R Discussion Paper Series 102, The Institute for Replication (I4R).
- Luigi Pace & Alessandra Salvan, 2020. "Likelihood, Replicability and Robbins' Confidence Sequences," International Statistical Review, International Statistical Institute, vol. 88(3), pages 599-615, December.
- Glenn Shafer, 2021. "Testing by betting: A strategy for statistical and scientific communication," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(2), pages 407-431, April.
- Strømland, Eirik, 2019. "Preregistration and reproducibility," Journal of Economic Psychology, Elsevier, vol. 75(PA).
- Giandomenica Becchio, 2020. "The Two Blades of Occam's Razor in Economics: Logical and Heuristic," Economic Thought, World Economics Association, vol. 9(1), pages 1-17, July.
- Julian Reiss, 2001. "Natural economic quantities and their measurement," Journal of Economic Methodology, Taylor & Francis Journals, vol. 8(2), pages 287-311.
- Matthew Rosenblatt & Link Tejavibulya & Rongtao Jiang & Stephanie Noble & Dustin Scheinost, 2024. "Data leakage inflates prediction performance in connectome-based machine learning models," Nature Communications, Nature, vol. 15(1), pages 1-15, December.
- Elbæk, Christian T. & Lystbæk, Martin Nørhede & Mitkidis, Panagiotis, 2022. "On the psychology of bonuses: The effects of loss aversion and Yerkes-Dodson law on performance in cognitively and mechanically demanding tasks," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 98(C).
- Bettina Bert & Céline Heinl & Justyna Chmielewska & Franziska Schwarz & Barbara Grune & Andreas Hensel & Matthias Greiner & Gilbert Schönfelder, 2019. "Refining animal research: The Animal Study Registry," PLOS Biology, Public Library of Science, vol. 17(10), pages 1-12, October.
- Aumann, Craig A., 2007. "A methodology for developing simulation models of complex systems," Ecological Modelling, Elsevier, vol. 202(3), pages 385-396.
- Florian Ellsaesser & Eric W. K. Tsang & Jochen Runde, 2014. "Models of causal inference: Imperfect but applicable is better than perfect but inapplicable," Strategic Management Journal, Wiley Blackwell, vol. 35(10), pages 1541-1551, October.
- Stephen Pratten, 2007. "Realism, closed systems and abstraction," Journal of Economic Methodology, Taylor & Francis Journals, vol. 14(4), pages 473-497.
- Kinouchi, Renato, 2018. "Philosophical issues related to risks and values," LSE Research Online Documents on Economics 90470, London School of Economics and Political Science, LSE Library.
- Bertoldi, Paolo & Mosconi, Rocco, 2020. "Do energy efficiency policies save energy? A new approach based on energy policy indicators (in the EU Member States)," Energy Policy, Elsevier, vol. 139(C).
- Maier, Maximilian & VanderWeele, Tyler & Mathur, Maya B, 2021. "Using Selection Models to Assess Sensitivity to Publication Bias: A Tutorial and Call for More Routine Use," MetaArXiv tp45u, Center for Open Science.
- Marcel Boumans & Mary Morgan, 2002. "Ceteris paribus conditions: materiality and the application of economic theories," Journal of Economic Methodology, Taylor & Francis Journals, vol. 8(1), pages 11-26.
More about this item
NEP fields
This paper has been announced in the following NEP Reports:- NEP-HME-2022-06-13 (Heterodox Microeconomics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:yxba5. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.