IDEAS home Printed from https://ideas.repec.org/p/zbw/i4rdps/68.html
   My bibliography  Save this paper

Is Economics Self-Correcting? Replications in the American Economic Review

Author

Listed:
  • Ankel-Peters, Jörg
  • Fiala, Nathan
  • Neubauer, Florian

Abstract

This paper reviews the impact of replications published as comments in the American Economic Review between 2010 and 2020. We examine their citations and influence on the original papers' subsequent citations. Our results show that comments are barely cited, and they do not affect the original paper's citations - even if the comment diagnoses substantive problems. Furthermore, we conduct an opinion survey among replicators and authors and find that there often is no consensus on whether the original paper's contribution sustains. We conclude that the economics literature does not self-correct, and that robustness and replicability are hard to define in economics.

Suggested Citation

  • Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Is Economics Self-Correcting? Replications in the American Economic Review," I4R Discussion Paper Series 68, The Institute for Replication (I4R).
    • Handle: RePEc:zbw:i4rdps:68
    as

    Download full text from publisher

    File URL: https://www.econstor.eu/bitstream/10419/276962/1/I4R-DP068.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. David Y. Albouy, 2012. "The Colonial Origins of Comparative Development: An Empirical Investigation: Comment," American Economic Review, American Economic Association, vol. 102(6), pages 3059-3076, October.
    2. Daron Acemoglu & Simon Johnson & James A. Robinson & Pierre Yared, 2008. "Income and Democracy," American Economic Review, American Economic Association, vol. 98(3), pages 808-842, June.
    3. Lucas C. Coffman & Muriel Niederle & Alistair J. Wilson, 2017. "A Proposal to Organize and Promote Replications," American Economic Review, American Economic Association, vol. 107(5), pages 41-45, May.
    4. Derek Lemoine & Ivan Rudik, 2017. "Steering the Climate System: Using Inertia to Lower the Cost of Policy," American Economic Review, American Economic Association, vol. 107(10), pages 2947-2957, October.
    5. H. M. Collins, 1991. "The Meaning of Replication and the Science of Economics," History of Political Economy, Duke University Press, vol. 23(1), pages 123-142, Spring.
    6. Jason Long & Joseph Ferrie, 2013. "Intergenerational Occupational Mobility in Great Britain and the United States since 1850: Reply," American Economic Review, American Economic Association, vol. 103(5), pages 2041-2049, August.
    7. Daniel S. Hamermesh, 2007. "Replication in Economics," NBER Working Papers 13026, National Bureau of Economic Research, Inc.
    8. Daron Acemoglu & Simon Johnson & James A. Robinson, 2001. "The Colonial Origins of Comparative Development: An Empirical Investigation," American Economic Review, American Economic Association, vol. 91(5), pages 1369-1401, December.
    9. Michael Hout & Avery M. Guest, 2013. "Intergenerational Occupational Mobility in Great Britain and the United States since 1850: Comment," American Economic Review, American Economic Association, vol. 103(5), pages 2021-2040, August.
    10. Whittington, Leslie A & Alm, James & Peters, H Elizabeth, 1990. "Fertility and the Personal Exemption: Implicit Pronatalist Policy in the United States," American Economic Review, American Economic Association, vol. 80(3), pages 545-556, June.
    11. Richard Crump & Gopi Shah Goda & Kevin J. Mumford, 2011. "Fertility and the Personal Exemption: Comment," American Economic Review, American Economic Association, vol. 101(4), pages 1616-1628, June.
    12. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    13. Jason Long & Joseph Ferrie, 2013. "Intergenerational Occupational Mobility in Great Britain and the United States since 1850," American Economic Review, American Economic Association, vol. 103(4), pages 1109-1137, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rose, Julian & Neubauer, Florian & Ankel-Peters, Jörg, 2024. "Long-term effects of the targeting the ultra-poor program: A reproducibility and replicability assessment of Banerjee et al. (2021)," Ruhr Economic Papers 1107, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    2. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    3. Tom Coupé & Thomas Logchies & W. Robert Reed, 2024. "Do Replications Really Receive Fewer Citations?," Working Papers in Economics 24/15, University of Canterbury, Department of Economics and Finance.
    4. Ankel-Peters, Jörg & Schmidt, Christoph M., 2023. "Rural electrification, the credibility revolution, and the limits of evidence-based policy," Ruhr Economic Papers 1051, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    5. Thibaut Arpinon & Marianne Lefebvre, 2024. "Registered Reports and Associated Benefits for Agricultural Economics," Post-Print hal-04635986, HAL.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Casey, Gregory & Klemp, Marc, 2016. "Instrumental Variables in the Long Run," MPRA Paper 68696, University Library of Munich, Germany.
    2. Markus Jäntti & Stephen P. Jenkins, 2013. "Income Mobility," SOEPpapers on Multidisciplinary Panel Data Research 607, DIW Berlin, The German Socio-Economic Panel (SOEP).
    3. David S. Johnson & Catherine Massey & Amy O’Hara, 2015. "The Opportunities and Challenges of Using Administrative Data Linkages to Evaluate Mobility," The ANNALS of the American Academy of Political and Social Science, , vol. 657(1), pages 247-264, January.
    4. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    5. Saima Nawaz & M. Idrees Khawaja, 2016. "Fiscal Policy, Institutions And Growth: New Insights," The Singapore Economic Review (SER), World Scientific Publishing Co. Pte. Ltd., vol. 64(05), pages 1251-1278, December.
    6. Zachary Ward, 2023. "Intergenerational Mobility in American History: Accounting for Race and Measurement Error," American Economic Review, American Economic Association, vol. 113(12), pages 3213-3248, December.
    7. Mußhoff, Oliver & Ölkers, Tim & Kirchner, Ella, 2024. "A note on the discussion regarding terrorism and land use in agriculture," Land Use Policy, Elsevier, vol. 144(C).
    8. Sen, Kunal & Pritchett, Lant & Kar, Sabyasachi & Raihan, Selim, 2016. "Democracy Versus Dictatorship? The Political Determinants of Growth Episodes," Working Paper Series rwp17-009, Harvard University, John F. Kennedy School of Government.
    9. Natkhov, T. & Polishchuk, L., 2017. "Political Economy of Institutions and Development: The Importance of Being Inclusive. Reflection on "Why Nations Fail" by D. Acemoglu and J. Robinson. Part I. Institutions and Economic Devel," Journal of the New Economic Association, New Economic Association, vol. 34(2), pages 12-38.
    10. Casey, Gregory & Klemp, Marc, 2021. "Historical instruments and contemporary endogenous regressors," Journal of Development Economics, Elsevier, vol. 149(C).
    11. Ran Abramitzky, 2015. "Economics and the Modern Economic Historian," NBER Working Papers 21636, National Bureau of Economic Research, Inc.
    12. Ang, James B., 2013. "Institutions and the long-run impact of early development," Journal of Development Economics, Elsevier, vol. 105(C), pages 1-18.
    13. Peter Lloyd & Cassey Lee, 2018. "A Review Of The Recent Literature On The Institutional Economics Analysis Of The Long†Run Performance Of Nations," Journal of Economic Surveys, Wiley Blackwell, vol. 32(1), pages 1-22, February.
    14. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    15. Dribe, Martin & Eriksson, Björn & Scalone, Francesco, 2019. "Migration, marriage and social mobility: Women in Sweden 1880–1900," Explorations in Economic History, Elsevier, vol. 71(C), pages 93-111.
    16. Fedderke, Johannes & Klitgaard, Robert, 2013. "How Much Do Rights Matter?," World Development, Elsevier, vol. 51(C), pages 187-206.
    17. Nina Boberg-Fazlić & Paul Sharp, 2018. "North and south: long-run social mobility in England and attitudes toward welfare," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 12(2), pages 251-276, May.
    18. Peter Lloyd & Cassey Lee, 2016. "A Review of the Recent Literature on the Institutional Economics Analysis of the Long-Run Performance of Nations," Department of Economics - Working Papers Series 2019, The University of Melbourne.
    19. Chakraborty, Adrij, 2017. "Colonial Origins and Comparative Development: Institutions Matter," MPRA Paper 86320, University Library of Munich, Germany, revised Feb 2018.
    20. Hoyt Bleakley & Joseph Ferrie, 2016. "Shocking Behavior: Random Wealth in Antebellum Georgia and Human Capital Across Generations," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(3), pages 1455-1495.

    More about this item

    Keywords

    replication; citations; meta-science;
    All these keywords.

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • B40 - Schools of Economic Thought and Methodology - - Economic Methodology - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:zbw:i4rdps:68. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: ZBW - Leibniz Information Centre for Economics (email available below). General contact details of provider: https://www.i4replication.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.