A scoping review on metrics to quantify reproducibility: a multitude of questions leads to a multitude of metrics
Author
Abstract
Suggested Citation
DOI: 10.31219/osf.io/apdxk
Download full text from publisher
References listed on IDEAS
- Leonhard Held, 2020. "A new standard for the analysis and design of replication studies," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 183(2), pages 431-448, February.
- Hanousek, Jan & Hajkova, Dana & Filer, Randall K., 2008.
"A rise by any other name? Sensitivity of growth regressions to data source,"
Journal of Macroeconomics, Elsevier, vol. 30(3), pages 1188-1206, September.
- Randall Filer & Jan Hanousek & Dana Hajkova, 2007. "A Rise By Any Other Name? Sensitivity of Growth Regressions to Data Source," William Davidson Institute Working Papers Series wp889, William Davidson Institute at the University of Michigan.
- Randall Filer & Dana Hajkova & Jan Hanousek, 2007. "A Rise by Any Other Name? Sensitivity of Growth Regressions to Data Source," CESifo Working Paper Series 2064, CESifo.
- Andrew C. Chang & Phillip Li, 2022. "Is Economics Research Replicable? Sixty Published Papers From Thirteen Journals Say “Often Notâ€," Critical Finance Review, now publishers, vol. 11(1), pages 185-206, February.
- Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021.
"The influence of hidden researcher decisions in applied microeconomics,"
Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily A. & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnit, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," IZA Discussion Papers 13233, Institute of Labor Economics (IZA).
- Huntington-Klein, Nick & Arenas, Andreu & Beam, Emily & Bertoni, Marco & Bloem, Jeffrey R. & Burli, Pralhad & Chen, Naibin & Greico, Paul & Ekpe, Godwin & Pugatch, Todd & Saavedra, Martin & Stopnitzky, 2020. "The Influence of Hidden Researcher Decisions in Applied Microeconomics," GLO Discussion Paper Series 537, Global Labor Organization (GLO).
- Francesco Pauli, 2019. "A Statistical Model to Investigate the Reproducibility Rate Based on Replication Experiments," International Statistical Review, International Statistical Institute, vol. 87(1), pages 68-79, April.
Most related items
These are the items that most often cite the same works as this one and are cited by the same works as this one.- Heyard, Rachel & Pawel, Samuel & Frese, Joris & Voelkl, Bernhard & Würbel, Hanno & McCann, Sarah & Held, Leonhard & Wever, Kimberley E. PhD & Hartmann, Helena & Townsin, Louise, 2024. "A scoping review on metrics to quantify reproducibility: a multitude of questions leads to a multitude of metrics," MetaArXiv apdxk_v1, Center for Open Science.
- Klege, Rebecca A. & Amuakwa-Mensah, Franklin & Visser, Martine, 2022. "Tenancy and energy choices in Rwanda. A replication and extension study," World Development Perspectives, Elsevier, vol. 26(C).
- Ankel-Peters, Jörg & Vance, Colin & Bensch, Gunther, 2022. "Spotlight on researcher decisions – Infrastructure evaluation, instrumental variables, and first-stage specification screening," OSF Preprints sw6kd_v1, Center for Open Science.
- Freuli, Francesca & Held, Leonhard & Heyard, Rachel, 2022. "Replication success under questionable research practices – a simulation study," MetaArXiv s4b65_v1, Center for Open Science.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad‐Díaz & Menachem (Meni) Abudy , 2024.
"Nonstandard Errors,"
Journal of Finance, American Finance Association, vol. 79(3), pages 2339-2390, June.
- Albert J. Menkveld & Anna Dreber & Félix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard, 2021. "Non-Standard Errors," Documents de travail du Centre d'Economie de la Sorbonne 21033, Université Panthéon-Sorbonne (Paris 1), Centre d'Economie de la Sorbonne.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Mena, 2021. "Non-Standard Errors," Working Papers 2021:17, Lund University, Department of Economics.
- Utz Weitzel & Michael Razen & Sebastian Neussüs & Michael Kirchler & Magnus Johannesson & Juergen Huber & Felix Holzmeister & Anna Dreber & Albert J. Menkveld & Javier Gil-Bazo, 2021. "Non-Standard Errors," Working Papers 1303, Barcelona School of Economics.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," IWH Discussion Papers 11/2021, Halle Institute for Economic Research (IWH).
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neussüs & Michael Razen & Utz Weitzel & Christian T. Brownlees & Javier Gil-Baz, 2021. "Non-standard errors," Economics Working Papers 1807, Department of Economics and Business, Universitat Pompeu Fabra.
- Albert J. et al. Menkveld, 2021. "Non-Standard Errors," CESifo Working Paper Series 9453, CESifo.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Post-Print halshs-03500882, HAL.
- Francesco Franzoni & Roxana Mihet & Markus Leippold & Per Ostberg & Olivier Scaillet & Norman Schürhoff & Oksana Bashchenko & Nicola Mano & Michele Pelli, 2022. "Non-Standard Errors," Swiss Finance Institute Research Paper Series 22-09, Swiss Finance Institute.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Edwin Baidoo & Michael Frömmel & et al, 2021. "Non-Standard Errors," Working Papers of Faculty of Economics and Business Administration, Ghent University, Belgium 21/1032, Ghent University, Faculty of Economics and Business Administration.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Hasse, Jean-Baptiste & e.a.,, 2023. "Non-Standard Errors," LIDAM Reprints LFIN 2023002, Université catholique de Louvain, Louvain Finance (LFIN).
- Moinas, Sophie & Declerck, Fany & Menkveld, Albert J. & Dreber, Anna, 2023. "Non-Standard Errors," TSE Working Papers 23-1451, Toulouse School of Economics (TSE).
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüß, Sebastian & Razen, Michael & Weitzel, Utz & Abad-Díaz, David & Abudy, Menac, 2024. "Nonstandard errors," LSE Research Online Documents on Economics 123002, London School of Economics and Political Science, LSE Library.
- Albert Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüß & Michael Razen & Utz Weitzel & David Abad-Díaz & Tobias Adrian & Yacine Ai, 2024. "Nonstandard Errors," Post-Print hal-04676112, HAL.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Cambridge Working Papers in Economics 2182, Faculty of Economics, University of Cambridge.
- Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Jürgen & Johannesson, Magnus & Kirchler, Michael & Neusüss, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-standard errors," SAFE Working Paper Series 327, Leibniz Institute for Financial Research SAFE.
- Albert J. Menkveld & Anna Dreber & Felix Holzmeister & Jürgen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & David Abad-Díaz & Menachem Abudy & Tobi, 2021. "Non-Standard Errors," Working Papers 2021-31, Faculty of Economics and Statistics, Universität Innsbruck.
- Ferrara, Gerardo & Jurkatis, Simon, 2021. "Non-standard errors," Bank of England working papers 955, Bank of England.
- Ciril Bosch-Rosa & Bernhard Kassner, 2023. "Non-Standard Errors," Rationality and Competition Discussion Paper Series 385, CRC TRR 190 Rationality and Competition.
- Albert J Menkveld & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johannesson & Michael Kirchler & Sebastian Neusüss & Michael Razen & Utz Weitzel & Gunther Capelle-Blancard & David Abad-Dí, 2021. "Non-Standard Errors," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-03500882, HAL.
- Menkveld, A. & Dreber, A. & Holzmeister, F. & Huber, J. & Johannesson, M. & Kirchler, M. & Neusüss, S. & Razen, M. & Neusüss, S. & Neusüss, S., 2021. "Non-Standard Errors," Janeway Institute Working Papers 2112, Faculty of Economics, University of Cambridge.
- Wolff, Christian & Menkveld, Albert J. & Dreber, Anna & Holzmeister, Felix & Huber, Juergen & Johannesson, Magnus & Kirchler, Michael & Neusüess, Sebastian & Razen, Michael & Weitzel, Utz, 2021. "Non-Standard Errors," CEPR Discussion Papers 16751, C.E.P.R. Discussion Papers.
- Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024.
"Long-Term Effects of the Targeting the Ultra-Poor Program - A Reproducibility and Replicability Assessment of Banerjee et al. (2021),"
I4R Discussion Paper Series
142, The Institute for Replication (I4R).
- Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-term effects of the targeting the ultra-poor program: A reproducibility and replicability assessment of Banerjee et al. (2021)," Ruhr Economic Papers 1107, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
- Randall K. Filer & Dragana Stanišić, 2016.
"The Effect of Terrorist Incidents on Capital Flows,"
Review of Development Economics, Wiley Blackwell, vol. 20(2), pages 502-513, May.
- Randall Filer & Dragana Stanišic, 2012. "The Effect of Terrorist Incidents on Capital Flows," CESifo Working Paper Series 3998, CESifo.
- Randall K. Filer & Dragana Stanisic, 2013. "The Effect of Terrorist Incidents on Capital Flows," CERGE-EI Working Papers wp480, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
- Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P & Hepplewhite, Matthew & de la Guardia, Fernando Hoces, 2024.
"Promoting Reproducibility and Replicability in Political Science,"
Department of Economics, Working Paper Series
qt23n3n3dg, Department of Economics, Institute for Business and Economic Research, UC Berkeley.
- Brodeur, Abel & Esterling, Kevin & Ankel-Peters, Jörg & Bueno, Natália S. & Desposato, Scott & Dreber, Anna & Genovese, Federica & Green, Donald P. & Hepplewhite, Matthew & Hoces de la Guardia, Fernan, 2024. "Promoting Reproducibility and Replicability in Political Science," I4R Discussion Paper Series 100, The Institute for Replication (I4R).
- Schnabel, Isabel & Körner, Tobias, 2010.
"Public Ownership of Banks and Economic Growth - The Role of Heterogeneity,"
CEPR Discussion Papers
8138, C.E.P.R. Discussion Papers.
- Tobias Körner & Isabel Schnabel, 2010. "Public Ownership of Banks and Economic Growth – The Role of Heterogeneity," Discussion Paper Series of the Max Planck Institute for Research on Collective Goods 2010_41, Max Planck Institute for Research on Collective Goods.
- Christophe Pérignon & Olivier Akmansoy & Christophe Hurlin & Anna Dreber & Felix Holzmeister & Juergen Huber & Magnus Johanneson & Michael Kirchler & Albert Menkveld & Michael Razen & Utz Weitzel, 2022. "Reproducibility of Empirical Results: Evidence from 1,000 Tests in Finance," Working Papers hal-03810013, HAL.
- Joakim A. Weill & Matthieu Stigler & Olivier Deschenes & Michael R. Springborn, 2021. "Researchers' Degrees-of-Flexibility and the Credibility of Difference-in-Differences Estimates: Evidence From the Pandemic Policy Evaluations," NBER Working Papers 29550, National Bureau of Economic Research, Inc.
- Andrew C. Chang & Trace J. Levinson, 2023. "Raiders of the lost high‐frequency forecasts: New data and evidence on the efficiency of the Fed's forecasting," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 38(1), pages 88-104, January.
- Petar Stankov, 2018.
"Deregulation, Economic Growth and Growth Acceleration,"
Journal of Economic Development, Chung-Ang Unviersity, Department of Economics, vol. 43(4), pages 21-40, December.
- Petar Stankov, 2010. "Deregulation, Economic Growth and Growth Acceleration," CERGE-EI Working Papers wp424, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
- Stankov, Petar, 2010. "Deregulation, economic growth and growth acceleration," MPRA Paper 26485, University Library of Munich, Germany.
- Muradchanian, Jasmine & Hoekstra, Rink & Kiers, Henk & van Ravenzwaaij, Don, 2020. "How Best to Quantify Replication Success? A Simulation Study on the Comparison of Replication Success Metrics," MetaArXiv wvdjf, Center for Open Science.
- Jong-A-Pin, Richard & Mierau, Jochen O., 2022.
"No country for old men: Aging dictators and economic growth,"
Economic Modelling, Elsevier, vol. 107(C).
- Jong-A-Pin, R. & Mierau, J. O., 2011. "No Country for Old Men: Aging Dictators and Economic Growth," Cambridge Working Papers in Economics 1158, Faculty of Economics, University of Cambridge.
- Richard Jong-A-Pin & Jochen O. Mierau, 2011. "No Country for Old Men: Aging Dictators and Economic Growth," Working Papers CEB 11-039, ULB -- Universite Libre de Bruxelles.
- David Warner & Prasada Rao & William E. Griffiths & Duangkamon Chotikapanich, 2011. "Global Inequality: Levels and Trends, 1993-2005," Discussion Papers Series 436, School of Economics, University of Queensland, Australia.
- repec:grz:wpsses:2021-08 is not listed on IDEAS
- Ulaşan, Bülent, 2012. "Cross-country growth empirics and model uncertainty: An overview," Economics - The Open-Access, Open-Assessment E-Journal (2007-2020), Kiel Institute for the World Economy (IfW Kiel), vol. 6, pages 1-69.
- Thomas Gries & Manfred Kraft & Daniel Meierrieks, 2011.
"Financial deepening, trade openness and economic growth in Latin America and the Caribbean,"
Applied Economics, Taylor & Francis Journals, vol. 43(30), pages 4729-4739.
- Thomas Gries & Manfred Kraft & Daniel Meierrieks, 2008. "Financial Deepening, Trade Openness and Economic Growth in Latin America and the Caribbean," Working Papers CIE 17, Paderborn University, CIE Center for International Economics.
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2024.
"p-Hacking, Data type and Data-Sharing Policy,"
The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
- Abel Brodeur & Nikolai Cook & Carina Neisser, 2022. "P-Hacking, Data Type and Data-Sharing Policy," ECONtribute Discussion Papers Series 200, University of Bonn and University of Cologne, Germany.
- Brodeur, Abel & Cook, Nikolai & Neisser, Carina, 2022. "P-Hacking, Data Type and Data-Sharing Policy," IZA Discussion Papers 15586, Institute of Labor Economics (IZA).
- Gries, Thomas & Kraft, Manfred & Meierrieks, Daniel, 2009.
"Linkages Between Financial Deepening, Trade Openness, and Economic Development: Causality Evidence from Sub-Saharan Africa,"
World Development, Elsevier, vol. 37(12), pages 1849-1860, December.
- Thomas Gries & Manfred Kraft & Daniel Meierrieks, 2008. "Linkages between Financial Deepening,Trade Openness and Economic Development: Causality Evidence from Sub-Saharan Africa," Working Papers CIE 15, Paderborn University, CIE Center for International Economics.
More about this item
NEP fields
This paper has been announced in the following NEP Reports:- NEP-PPM-2024-12-30 (Project, Program and Portfolio Management)
- NEP-SOG-2024-12-30 (Sociology of Economics)
Statistics
Access and download statisticsCorrections
All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:osf:metaar:apdxk. See general information about how to correct material in RePEc.
If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.
If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .
If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.
For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: OSF (email available below). General contact details of provider: https://osf.io/preprints/metaarxiv .
Please note that corrections may take a couple of weeks to filter through the various RePEc services.