IDEAS home Printed from https://ideas.repec.org/p/osf/metaar/t93cg.html
   My bibliography  Save this paper

Toward More Transparency in Statistical Practice

Author

Listed:
  • Wagenmakers, Eric-Jan

    (University of Amsterdam)

  • Sarafoglou, Alexandra

    (University of Amsterdam)

  • Aarts, Sil Dr.

    (Maastricht University)

  • Albers, Casper J

    (University of Groningen)

  • Algermissen, Johannes

    (Radboud University Nijmegen)

  • Bahník, Štěpán

    (University of Economics, Prague)

  • van Dongen, Noah N'Djaye Nikolai
  • Hoekstra, Rink
  • Moreau, David
  • van Ravenzwaaij, Don

    (University of Groningen)

Abstract

We explore the promise of statistical reform by starting from the assumption that most researchers would endorse Merton's ethos of science as reflected in the four norms of communalism, universalism, disinterestedness, and organized skepticism. Translated to data analysis, these norms imply a need for transparency, a fair acknowledgement of uncertainty, and openness to alternative interpretations. We discuss seven statistical procedures, both old and new, that we believe can positively impact statistical practice in the social and behavioral sciences.

Suggested Citation

  • Wagenmakers, Eric-Jan & Sarafoglou, Alexandra & Aarts, Sil Dr. & Albers, Casper J & Algermissen, Johannes & Bahník, Štěpán & van Dongen, Noah N'Djaye Nikolai & Hoekstra, Rink & Moreau, David & van Rav, 2021. "Toward More Transparency in Statistical Practice," MetaArXiv t93cg, Center for Open Science.
    • Handle: RePEc:osf:metaar:t93cg
    DOI: 10.31219/osf.io/t93cg
    as

    Download full text from publisher

    File URL: https://osf.io/download/603e494967386c031361e531/
    Download Restriction: no
    File URL: https://libkey.io/10.31219/osf.io/t93cg?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gelman A. & Pasarica C. & Dodhia R., 2002. "Lets Practice What We Preach: Turning Tables into Graphs," The American Statistician, American Statistical Association, vol. 56, pages 121-130, May.
    2. Susan Athey & Guido W. Imbens, 2019. "Machine Learning Methods That Economists Should Know About," Annual Review of Economics, Annual Reviews, vol. 11(1), pages 685-725, August.
    3. Blakeley B. McShane & David Gal & Andrew Gelman & Christian Robert & Jennifer L. Tackett, 2019. "Abandon Statistical Significance," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 235-245, March.
    4. Levine, Ross & Renelt, David, 1992. "A Sensitivity Analysis of Cross-Country Growth Regressions," American Economic Review, American Economic Association, vol. 82(4), pages 942-963, September.
    5. Tracey L Weissgerber & Natasa M Milic & Stacey J Winham & Vesna D Garovic, 2015. "Beyond Bar and Line Graphs: Time for a New Data Presentation Paradigm," PLOS Biology, Public Library of Science, vol. 13(4), pages 1-10, April.
    6. Darren B Taichman & Peush Sahni & Anja Pinborg & Larry Peiperl & Christine Laine & Astrid James & Sung-Tae Hong & Abraham Haileamlak & Laragh Gollogly & Fiona Godlee & Frank A Frizelle & Fernando Flor, 2017. "Data Sharing Statements for Clinical Trials: A Requirement of the International Committee of Medical Journal Editors," PLOS Medicine, Public Library of Science, vol. 14(6), pages 1-3, June.
    7. Noah N. N. van Dongen & Johnny B. van Doorn & Quentin F. Gronau & Don van Ravenzwaaij & Rink Hoekstra & Matthias N. Haucke & Daniel Lakens & Christian Hennig & Richard D. Morey & Saskia Homer & Andrew, 2019. "Multiple Perspectives on Inference for Two Simple Statistical Scenarios," The American Statistician, Taylor & Francis Journals, vol. 73(S1), pages 328-339, March.
    8. Athey, Susan & Imbens, Guido W., 2019. "Machine Learning Methods Economists Should Know About," Research Papers 3776, Stanford University, Graduate School of Business.
    9. Uri Simonsohn & Joseph P. Simmons & Leif D. Nelson, 2020. "Specification curve analysis," Nature Human Behaviour, Nature, vol. 4(11), pages 1208-1214, November.
    10. Leamer, Edward E, 1985. "Sensitivity Analyses Would Help," American Economic Review, American Economic Association, vol. 75(3), pages 308-313, June.
    11. Jonah Gabry & Daniel Simpson & Aki Vehtari & Michael Betancourt & Andrew Gelman, 2019. "Visualization in Bayesian workflow," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 182(2), pages 389-402, February.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Bennett, Davara L. & Webb, Calum J.R. & Mason, Kate E. & Schlüter, Daniela K. & Fahy, Katie & Alexiou, Alexandros & Wickham, Sophie & Barr, Ben & Taylor-Robinson, David, 2021. "Funding for preventative Children’s Services and rates of children becoming looked after: A natural experiment using longitudinal area-level data in England," Children and Youth Services Review, Elsevier, vol. 131(C).
    2. Sacker, Amanda & Lacey, Rebecca E. & Maughan, Barbara & Murray, Emily T., 2022. "Out-of-home care in childhood and socio-economic functioning in adulthood: ONS Longitudinal study 1971–2011," Children and Youth Services Review, Elsevier, vol. 132(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Eric-Jan Wagenmakers & Alexandra Sarafoglou & Sil Aarts & Casper Albers & Johannes Algermissen & Štěpán Bahník & Noah Dongen & Rink Hoekstra & David Moreau & Don Ravenzwaaij & Aljaž Sluga & Franziska , 2021. "Seven steps toward more transparency in statistical practice," Nature Human Behaviour, Nature, vol. 5(11), pages 1473-1480, November.
    2. Guillaume Coqueret, 2023. "Forking paths in financial economics," Papers 2401.08606, arXiv.org.
    3. Gabriel Okasa & Kenneth A. Younge, 2022. "Sample Fit Reliability," Papers 2209.06631, arXiv.org.
    4. Sophie-Charlotte Klose & Johannes Lederer, 2020. "A Pipeline for Variable Selection and False Discovery Rate Control With an Application in Labor Economics," Papers 2006.12296, arXiv.org, revised Jun 2020.
    5. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP72/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.
    6. Ay, Jean-Sauveur & Le Gallo, Julie, 2021. "The Signaling Values of Nested Wine Names," Working Papers 321851, American Association of Wine Economists.
    7. Sai Ding & John Knight, 2011. "Why has China Grown So Fast? The Role of Physical and Human Capital Formation," Oxford Bulletin of Economics and Statistics, Department of Economics, University of Oxford, vol. 73(2), pages 141-174, April.
    8. Rok Spruk & Mitja Kovac, 2018. "Inefficient Growth," Review of Economics and Institutions, Università di Perugia, vol. 9(2).
    9. Chen, Ruoyu & Jiang, Hanchen & Quintero, Luis E., 2023. "Measuring the value of rent stabilization and understanding its implications for racial inequality: Evidence from New York City," Regional Science and Urban Economics, Elsevier, vol. 103(C).
    10. R Burger & S du Plessis, 2011. "Examining the Robustness of Competing Explanations of Slow Growth in African Countries," Studies in Economics and Econometrics, Taylor & Francis Journals, vol. 35(3), pages 21-47, December.
    11. Dangxing Chen & Luyao Zhang, 2023. "Monotonicity for AI ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance," Papers 2301.07060, arXiv.org.
    12. Ballestar, María Teresa & Mir, Miguel Cuerdo & Pedrera, Luis Miguel Doncel & Sainz, Jorge, 2024. "Effectiveness of tutoring at school: A machine learning evaluation," Technological Forecasting and Social Change, Elsevier, vol. 199(C).
    13. Carmen Fernandez & Eduardo Ley & Mark F. J. Steel, 2001. "Model uncertainty in cross-country growth regressions," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 16(5), pages 563-576.
    14. Daniel Levy & Tamir Mayer & Alon Raviv, 2020. "Academic Scholarship in Light of the 2008 Financial Crisis: Textual Analysis of NBER Working Papers," Working Papers hal-02488796, HAL.
    15. Combes, Pierre-Philippe & Gobillon, Laurent & Zylberberg, Yanos, 2022. "Urban economics in a historical perspective: Recovering data with machine learning," Regional Science and Urban Economics, Elsevier, vol. 94(C).
    16. Martin Gassebner & Jerg Gutmann & Stefan Voigt, 2016. "When to expect a coup d’état? An extreme bounds analysis of coup determinants," Public Choice, Springer, vol. 169(3), pages 293-313, December.
    17. Barzin,Samira & Avner,Paolo & Maruyama Rentschler,Jun Erik & O’Clery,Neave, 2022. "Where Are All the Jobs ? A Machine Learning Approach for High Resolution Urban Employment Prediction inDeveloping Countries," Policy Research Working Paper Series 9979, The World Bank.
    18. Arenas, Andreu & Calsamiglia, Caterina, 2022. "Gender Differences in High-Stakes Performance and College Admission Policies," IZA Discussion Papers 15550, Institute of Labor Economics (IZA).
    19. Tsang, Andrew, 2021. "Uncovering Heterogeneous Regional Impacts of Chinese Monetary Policy," MPRA Paper 110703, University Library of Munich, Germany.
    20. Kyle Colangelo & Ying-Ying Lee, 2019. "Double debiased machine learning nonparametric inference with continuous treatments," CeMMAP working papers CWP54/19, Centre for Microdata Methods and Practice, Institute for Fiscal Studies.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:t93cg. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.