IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp10403.html
   My bibliography  Save this paper

Replication in Labor Economics: Evidence from Data, and What It Suggests

Author

Listed:
  • Hamermesh, Daniel S.

    (University of Texas at Austin)

Abstract

Examining the most heavily-cited publications in labor economics from the early 1990s, I show that few of over 3000 articles citing them directly replicates them. They are replicated more frequently using data from other time periods and economies, so that the validity of their central ideas has typically been verified. This pattern of scholarship suggests, beyond the currently required depositing of data and code upon publication, that there is little need for formal mechanisms for replication. The market for scholarship already produces replications of non-laboratory applied research.

Suggested Citation

  • Hamermesh, Daniel S., 2016. "Replication in Labor Economics: Evidence from Data, and What It Suggests," IZA Discussion Papers 10403, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp10403
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp10403.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Joonkyung Ha & Jong-Wha Lee & Lea Sumulong, 2010. "Rebalancing Growth in the Republic of Korea," Working Papers id:3120, eSocialSciences.
    2. Card, David & Krueger, Alan B, 1994. "Minimum Wages and Employment: A Case Study of the Fast-Food Industry in New Jersey and Pennsylvania," American Economic Review, American Economic Association, vol. 84(4), pages 772-793, September.
    3. ., 2010. "A Review of the Literature on Growth," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 3, Edward Elgar Publishing.
    4. Steven J. Davis & John Haltiwanger, 1992. "Gross Job Creation, Gross Job Destruction, and Employment Reallocation," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 107(3), pages 819-863.
    5. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics, Canadian Economics Association, vol. 40(3), pages 715-733, August.
    6. Ashenfelter, Orley & Krueger, Alan B, 1994. "Estimates of the Economic Returns to Schooling from a New Sample of Twins," American Economic Review, American Economic Association, vol. 84(5), pages 1157-1173, December.
    7. ., 2010. "Warranted Growth and the Role of the State," Chapters, in: Strategic Competition, Dynamics, and the Role of the State, chapter 5, Edward Elgar Publishing.
    8. Thomas Herndon & Michael Ash & Robert Pollin, 2014. "Does high public debt consistently stifle economic growth? A critique of Reinhart and Rogoff," Cambridge Journal of Economics, Cambridge Political Economy Society, vol. 38(2), pages 257-279.
    9. Carmen M. Reinhart & Kenneth S. Rogoff, 2010. "Growth in a Time of Debt," American Economic Review, American Economic Association, vol. 100(2), pages 573-578, May.
    10. Edward P. Lazear, 1990. "Job Security Provisions and Employment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(3), pages 699-726.
    11. Daniel S. Hamermesh, 2007. "Replication in Economics," NBER Working Papers 13026, National Bureau of Economic Research, Inc.
    12. Neal, Derek A & Johnson, William R, 1996. "The Role of Premarket Factors in Black-White Wage Differences," Journal of Political Economy, University of Chicago Press, vol. 104(5), pages 869-895, October.
    13. Reinhart, Carmen & Rogoff, Kenneth, 2010. "Debt and Growth Revisited," MPRA Paper 24376, University Library of Munich, Germany.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Marianne Saam, 2021. "Überlegungen zu Open Science in der Volkswirtschaftslehre [Reflections on Open Science in Economics]," Wirtschaftsdienst, Springer;ZBW - Leibniz Information Centre for Economics, vol. 101(10), pages 746-747, October.
    2. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 48(1), pages 62-83.
    3. Hernández Alemán, Anastasia & León, Carmelo J., 2018. "La Réplica en el Análisis Económico Aplicado/Replication in Applied Economic Analysis," Estudios de Economia Aplicada, Estudios de Economia Aplicada, vol. 36, pages 317-332, Enero.
    4. Ankel-Peters, Jörg & Fiala, Nathan & Neubauer, Florian, 2023. "Do economists replicate?," Journal of Economic Behavior & Organization, Elsevier, vol. 212(C), pages 219-232.
    5. Christian Leuz, 2018. "Evidence-based policymaking: promise, challenges and opportunities for accounting and financial markets research," Accounting and Business Research, Taylor & Francis Journals, vol. 48(5), pages 582-608, July.
    6. Colliard, Jean-Edouard & Hurlin, Christophe & Pérignon, Christophe, 2019. "Reproducibility Certification in Economics Research," HEC Research Papers Series 1345, HEC Paris.
    7. Aditya Bhattacharjea, 2019. "Labour Market Flexibility in Indian Industry A Critical Survey of the Literature," Working papers 296, Centre for Development Economics, Delhi School of Economics.
    8. Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhuber, 2024. "Reproduce to validate: A comprehensive study on the reproducibility of economics research," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 57(3), pages 961-988, August.
    9. Kenneth Button, 2020. "Studying the empirical implications of the liberalization of airport markets," Competition and Regulation in Network Industries, , vol. 21(3), pages 223-243, September.
    10. Abel Brodeur & Nikolai Cook & Carina Neisser, 2024. "p-Hacking, Data type and Data-Sharing Policy," The Economic Journal, Royal Economic Society, vol. 134(659), pages 985-1018.
    11. Eszter Czibor & David Jimenez‐Gomez & John A. List, 2019. "The Dozen Things Experimental Economists Should Do (More of)," Southern Economic Journal, John Wiley & Sons, vol. 86(2), pages 371-432, October.
    12. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    13. Anderson, Richard G., 2017. "Should you choose to do so...: A replication paradigm," Economics Discussion Papers 2017-79, Kiel Institute for the World Economy (IfW Kiel).
    14. W. Robert Reed, 2017. "Replication in labor economics," IZA World of Labor, Institute of Labor Economics (IZA), pages 413-413, December.
    15. Nicolas Vallois & Dorian Jullien, 2017. "Replication in Experimental Economics: A Historical and Quantitative Approach Focused on Public Good Game Experiments," GREDEG Working Papers 2017-21, Groupe de REcherche en Droit, Economie, Gestion (GREDEG CNRS), Université Côte d'Azur, France.
    16. Josephson, Anna & Michler, Jeffrey D., 2018. "Viewpoint: Beasts of the field? Ethics in agricultural and applied economics," Food Policy, Elsevier, vol. 79(C), pages 1-11.
    17. Sylvérie Herbert & Hautahi Kingi & Flavio Stanchi & Lars Vilhubern, 2021. "The Reproducibility of Economics Research: A Case Study," Working papers 853, Banque de France.
    18. Travis Richardson & Georgios Nalbantis & Tim Pawlowski, 2023. "Emotional Cues and the Demand for Televised Sports: Evidence from the UEFA Champions League," Journal of Sports Economics, , vol. 24(8), pages 993-1025, December.
    19. Nicolas Vallois & Dorian Jullien, 2017. "Replication in experimental economics: A historical and quantitative approach focused on public good game experiments," Université Paris1 Panthéon-Sorbonne (Post-Print and Working Papers) halshs-01651080, HAL.
    20. Hensel, Przemysław G., 2021. "Reproducibility and replicability crisis: How management compares to psychology and economics – A systematic review of literature," European Management Journal, Elsevier, vol. 39(5), pages 577-594.
    21. Aditya BHATTACHARJEA, 2021. "Labour market flexibility in Indian manufacturing: A critical survey of the literature," International Labour Review, International Labour Organization, vol. 160(2), pages 197-217, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Consiglio Andrea & Zenios Stavros A., 2015. "Risk Management Optimization for Sovereign Debt Restructuring," Journal of Globalization and Development, De Gruyter, vol. 6(2), pages 181-213, December.
    2. Di Sanzo Silvestro & Bella Mariano, 2015. "Public debt and growth in the euro area: evidence from parametric and nonparametric Granger causality," The B.E. Journal of Macroeconomics, De Gruyter, vol. 15(2), pages 631-648, July.
    3. repec:rim:rimpre:15_01 is not listed on IDEAS
    4. Lorenzo Forni & Geremia Palomba & Joana Pereira & Christine Richmond, 2021. "Sovereign debt restructuring and growth," Oxford Economic Papers, Oxford University Press, vol. 73(2), pages 671-697.
    5. Gianluigi Pelloni & Marco Savioli, 2015. "Why Is Italy Doing So Badly?," Economic Affairs, Wiley Blackwell, vol. 35(3), pages 349-365, October.
    6. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    7. Nick Huntington‐Klein & Andreu Arenas & Emily Beam & Marco Bertoni & Jeffrey R. Bloem & Pralhad Burli & Naibin Chen & Paul Grieco & Godwin Ekpe & Todd Pugatch & Martin Saavedra & Yaniv Stopnitzky, 2021. "The influence of hidden researcher decisions in applied microeconomics," Economic Inquiry, Western Economic Association International, vol. 59(3), pages 944-960, July.
    8. Valérie Orozco & Christophe Bontemps & Élise Maigné & Virginie Piguet & Annie Hofstetter & Anne Marie Lacroix & Fabrice Levert & Jean-Marc Rousselle, 2017. "How to make a pie? Reproducible Research for Empirical Economics & Econometrics," Post-Print hal-01939942, HAL.
    9. Colliard, Jean-Edouard & Hurlin, Christophe & Pérignon, Christophe, 2019. "Reproducibility Certification in Economics Research," HEC Research Papers Series 1345, HEC Paris.
    10. Joseph E. Stiglitz, 2015. "Reconstructing Macroeconomic Theory to Manage Economic Policy," Palgrave Macmillan Books, in: Éloi Laurent & Jacques Cacheux (ed.), Fruitful Economics, chapter 1, pages 20-56, Palgrave Macmillan.
    11. Mueller-Langer, Frank & Fecher, Benedikt & Harhoff, Dietmar & Wagner, Gert G., 2019. "Replication studies in economics—How many and which papers are chosen for replication, and why?," Research Policy, Elsevier, vol. 48(1), pages 62-83.
    12. Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020. "How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
    13. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    14. Annette N. Brown & Drew B. Cameron & Benjamin D. K. Wood, 2014. "Quality evidence for policymaking: I'll believe it when I see the replication," Journal of Development Effectiveness, Taylor & Francis Journals, vol. 6(3), pages 215-235, September.
    15. Claudio Montenegro & Carmen Pagés, 2005. "Who Benefits from Labor Market Regulations? Chile 1960-1998," Central Banking, Analysis, and Economic Policies Book Series, in: Jorge Restrepo & Andrea Tokman R. & Norman Loayza (Series Editor) & Klaus Schmidt-Hebbel (Series Edi (ed.),Labor Markets and Institutions, edition 1, volume 8, chapter 4, pages 077-114, Central Bank of Chile.
    16. Markus Eberhardt & Andrea Filippo Presbitero, 2013. "This Time They're Different: Heterogeneity;and Nonlinearity in the Relationship;between Debt and Growth," Mo.Fi.R. Working Papers 92, Money and Finance Research group (Mo.Fi.R.) - Univ. Politecnica Marche - Dept. Economic and Social Sciences.
    17. Kazuo Nishimura & Carine Nourry & Thomas Seegmuller & Alain Venditti, 2015. "On the (de)stabilizing effect of public debt in a Ramsey model with heterogeneous agents," International Journal of Economic Theory, The International Society for Economic Theory, vol. 11(1), pages 7-24, March.
    18. Swamy, Vighneswara, 2015. "Government Debt and its Macroeconomic Determinants – An Empirical Investigation," MPRA Paper 64106, University Library of Munich, Germany.
    19. Mencinger, Jernej & Verbic, Miroslav & Aristovnik, Aleksander, 2015. "Revisiting the role of public debt in economic growth: The case of OECD countries," MPRA Paper 67704, University Library of Munich, Germany.
    20. Vighneswara Swamy, 2015. "Government Debt and Economic Growth:Estimating the Debt Thresholds and Debt Intolerance," IEG Working Papers 361, Institute of Economic Growth.
    21. Sebastian Gechert, 2022. "Reconsidering macroeconomic policy prescriptions with meta-analysis [Statistical nonsignificance in empirical economics]," Industrial and Corporate Change, Oxford University Press and the Associazione ICC, vol. 31(2), pages 576-590.

    More about this item

    Keywords

    scientific method; reliability of research; citation analysis;
    All these keywords.

    JEL classification:

    • B21 - Schools of Economic Thought and Methodology - - History of Economic Thought since 1925 - - - Microeconomics
    • J01 - Labor and Demographic Economics - - General - - - Labor Economics: General
    • B41 - Schools of Economic Thought and Methodology - - Economic Methodology - - - Economic Methodology

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp10403. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.