IDEAS home Printed from https://ideas.repec.org/a/aea/aecrev/v107y2017i5p37-40.html
   My bibliography  Save this article

Replication in Labor Economics: Evidence from Data, and What It Suggests

Author

Listed:
  • Daniel S. Hamermesh

Abstract

Examining the most heavily cited publications in labor economics from the early 1990s, I show that few of over 3,000 articles, citing them directly, replicates them. They are replicated more frequently using data from other time periods and economies, so that the validity of their central ideas has typically been verified. This pattern of scholarship suggests, beyond the currently required depositing of data and code upon publication, that there is little need for formal mechanisms for replication. The market for scholarship already produces replications of non-laboratory applied research.

Suggested Citation

  • Daniel S. Hamermesh, 2017. "Replication in Labor Economics: Evidence from Data, and What It Suggests," American Economic Review, American Economic Association, vol. 107(5), pages 37-40, May.
    • Handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:37-40
    Note: DOI: 10.1257/aer.p20171121
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/articles?id=10.1257/aer.p20171121
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=3vb1hfCYXNKRTzIegTQxH7sGL-MNGTzG
    Download Restriction: no
    File URL: https://www.aeaweb.org/articles/attachments?retrieve=Xv2xj2aaR35Hu_x9ShnpjZjmFa5rKIkI
    Download Restriction: Access to full text is restricted to AEA members and institutional subscribers.
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. ., 2010. "Warranted Growth and the Role of the State," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 5, Edward Elgar Publishing.
    2. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 38(2), pages 257-279.
    3. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    4. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    5. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    6. Reinhart, Carmen & Rogoff, Kenneth, 2010. "Debt and Growth Revisited," MPRA Paper 24376, University Library of Munich, Germany.
    7. Steven J. Davis & John Haltiwanger, 1992. "Gross Job Creation, Gross Job Destruction, and Employment Reallocation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 819-863.
    8. Edward P. Lazear, 1990. "Job Security Provisions and Employment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(3), pages 699-726.
    9. ., 2010. "A Review of the Literature on Growth," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 3, Edward Elgar Publishing.
    10. Joonkyung Ha & Jong-Wha Lee & Lea Sumulong, 2010. "Rebalancing Growth in the Republic of Korea," Macroeconomics Working Papers 22753, East Asian Bureau of Economic Research.
    11. Ashenfelter, Orley & Krueger, Alan B, 1994. "Estimates of the Economic Returns to Schooling from a New Sample of Twins," American Economic Review, American Economic Association, vol. 84(5), pages 1157-1173, December.
    12. Hamermesh, Daniel S., 2007. "Replication in Economics," IZA Discussion Papers 2760, Institute of Labor Economics (IZA).
    13. Neal, Derek A & Johnson, William R, 1996. "The Role of Premarket Factors in Black-White Wage Differences," Journal of Political Economy, University of Chicago Press, vol. 104(5), pages 869-895, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marianne Saam, 2021. "Überlegungen zu Open Science in der Volkswirtschaftslehre [Reflections on Open Science in Economics]," Wirtschaftsdienst, Springer;ZBW - Leibniz Information Centre for Economics, vol. 101(10), pages 746-747, October.
    2. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
    3. Travis Richardson & Georgios Nalbantis & Tim Pawlowski, 2023. "Emotional Cues and the Demand for Televised Sports: Evidence from the UEFA Champions League," Journal of Sports Economics, , vol. 24(8), pages 993-1025, December.
    4. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
    5. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    6. Christian Leuz, 2018. "Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research," Accounting and Business Research, Taylor & Francis Journals, vol. 48(5), pages 582-608, July.
    7. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
    8. Aditya Bhattacharjea, 2019. "Labour Market Flexibility in Indian Industry A Critical Survey of the Literature," Working papers 296, Centre for Development Economics, Delhi School of Economics.
    9. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    10. Aditya BHATTACHARJEA, 2021. "Labour market flexibility in Indian manufacturing: A critical survey of the literature," International Labour Review, International Labour Organization, vol. 160(2), pages 197-217, June.
    11. Kenneth Button, 2020. "Studying the empirical implications of the liberalization of airport markets," Competition and Regulation in Network Industries, , vol. 21(3), pages 223-243, September.
    12. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    13. W. Robert Reed, 2017. "Replication in labor economics," IZA World of Labor, Institute of Labor Economics (IZA), pages 413-413, December.
    14. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    15. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    16. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    17. Anderson, Richard G., 2017. "Should you choose to do so...: A replication paradigm," Economics Discussion Papers 2017-79, Kiel Institute for the World Economy (IfW Kiel).
    18. Nicolas Vallois & Dorian Jullien, 2017. "Replication in Experimental Economics: A Historical and Quantitative Approach Focused on Public Good Game Experiments," GREDEG Working Papers 2017-21, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    19. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    20. Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Consiglio Andrea & Zenios Stavros A., 2015. "Risk Management Optimization for Sovereign Debt Restructuring," Journal of Globalization and Development, De Gruyter, vol. 6(2), pages 181-213, December.
    2. Gianluigi Pelloni & Marco Savioli, 2015. "Why Is Italy Doing So Badly?," Economic Affairs, Wiley Blackwell, vol. 35(3), pages 349-365, October.
    3. Di Sanzo Silvestro & Bella Mariano, 2015. "Public debt and growth in the euro area: evidence from parametric and nonparametric Granger causality," The B.E. Journal of Macroeconomics, De Gruyter, vol. 15(2), pages 631-648, July.
    4. repec:rim:rimpre:15_01 is not listed on IDEAS
    5. Lorenzo Forni & Geremia Palomba & Joana Pereira & Christine Richmond, 2021. "Sovereign debt restructuring and growth," Oxford Economic Papers, Oxford University Press, vol. 73(2), pages 671-697.
    6. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    9. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    10. Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    11. Christophe Hurlin & Christophe Pérignon, 2020. "Reproducibility Certification in Economics Research," Working Papers hal-02896404, HAL.
    12. Joseph E. Stiglitz, 2015. "Reconstructing Macroeconomic Theory to Manage Economic Policy," Palgrave Macmillan Books, in: Éloi Laurent & Jacques Cacheux (ed.), Fruitful Economics, chapter 1, pages 20-56, Palgrave Macmillan.
    13. Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020. "How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
    14. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    15. Daniel S. Hamermesh & Katie R. Genadek & Michael C. Burda, 2022. "Reply to “Racial Differences in Time at Work Not Working†by William A. Darity Jr. et al," ILR Review, Cornell University, ILR School, vol. 75(3), pages 573-577, May.
    16. Claudio Montenegro & Carmen Pagés, 2005. "Who Benefits from Labor Market Regulations? Chile 1960-1998," Central Banking, Analysis, and Economic Policies Book Series, in: Jorge Restrepo & Andrea Tokman R. & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series Edi (ed.),Labor Markets and Institutions, edition 1, volume 8, chapter 4, pages 077-114, Central Bank of Chile.
    17. Markus Eberhardt & Andrea Filippo Presbitero, 2013. "This Time They're Different: Heterogeneity;and Nonlinearity in the Relationship;between Debt and Growth," Mo.Fi.R. Working Papers 92, Money and Finance Research group (Mo.Fi.R.) - Univ. Politecnica Marche - Dept. Economic and Social Sciences.
    18. Paulo André Camuri & Frederico G. Jayme Jr. & Ana Maria Hermeto, 2015. "Fiscal consolidation in developed and emerging economies [Fiscal consolidation in developed and emerging economies]," Nova Economia, Economics Department, Universidade Federal de Minas Gerais (Brazil), vol. 25(spe), pages 835-861, December.
    19. Kazuo Nishimura & Carine Nourry & Thomas Seegmuller & Alain Venditti, 2015. "On the (de)stabilizing effect of public debt in a Ramsey model with heterogeneous agents," International Journal of Economic Theory, The International Society for Economic Theory, vol. 11(1), pages 7-24, March.
    20. Swamy, Vighneswara, 2015. "Government Debt and Economic Growth – Decomposing the Cause and Effect Relationship," MPRA Paper 64105, University Library of Munich, Germany.
    21. Swamy, Vighneswara, 2015. "Government Debt and its Macroeconomic Determinants – An Empirical Investigation," MPRA Paper 64106, University Library of Munich, Germany.

    More about this item

    JEL classification:

    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics
    • J00 - Labor and Demographic Economics - - General - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:aecrev:v:107:y:2017:i:5:p:37-40. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.