IDEAS home Printed from https://ideas.repec.org/a/wly/canjec/v41y2008i4p1406-1420.html
   My bibliography  Save this article

Do economics journal archives promote replicable research?

Author

Listed:
  • B.D. McCullough
  • Kerry Anne McGeary
  • Teresa D. Harrison

Abstract

. All the long‐standing archives at economics journals do not facilitate the reproduction of published results. The data‐only archives at Journal of Business and Economic Statistics and Economic Journal fail in part because most authors do not contribute data. Results published in the FRB St. Louis Review can rarely be reproduced using the data+code in the journal archive. Recently created archives at top journals should avoid the mistakes of their predecessors. We categorize reasons for archives' failures and identify successful policies. Toutes les archives de longue date des revues économiques ne facilitent pas la réplication des résultats publiés. Les archives de données du Journal of Business and Economic Statistics, et du Economic Journal achoppent en partie parce que la plupart des auteurs ne leur confient pas leurs données. Les résultats publiés dans la FRB St. Louis Review peuvent rarement être répliqués en utilisant les données et codes des archives de la revue. Les archives créées récemment par les meilleures revues devraient pouvoir éviter les erreurs de leurs prédécesseurs. On catégorise les raisons pour lesquelles les archives n'ont pas bien joué leur rôle et on identifie les politiques plus prometteuses.

Suggested Citation

  • B.D. McCullough & Kerry Anne McGeary & Teresa D. Harrison, 2008. "Do economics journal archives promote replicable research?," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 41(4), pages 1406-1420, November.
    • Handle: RePEc:wly:canjec:v:41:y:2008:i:4:p:1406-1420
    DOI: 10.1111/j.1540-5982.2008.00509.x
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/j.1540-5982.2008.00509.x
    Download Restriction: no
    File URL: https://libkey.io/10.1111/j.1540-5982.2008.00509.x?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Steven D. Levitt, 2002. "Using Electoral Cycles in Police Hiring to Estimate the Effects of Police on Crime: Reply," American Economic Review, American Economic Association, vol. 92(4), pages 1244-1250, September.
    2. Caroline M. Hoxby, 2000. "Does Competition among Public Schools Benefit Students and Taxpayers?," American Economic Review, American Economic Association, vol. 90(5), pages 1209-1238, December.
    3. John J. Donohue III & Steven D. Levitt, 2001. "The Impact of Legalized Abortion on Crime," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 116(2), pages 379-420.
    4. Hamermesh, Daniel S., 1997. "Some thoughts on replications and reviews," Labour Economics, Elsevier, vol. 4(2), pages 107-109, June.
    5. Christopher L. Foote & Christopher F. Goetz, 2005. "Testing economic hypotheses with state-level data: a comment on Donohue and Levitt (2001)," Working Papers 05-15, Federal Reserve Bank of Boston.
    6. Felix Oberholzer-Gee & Koleman Strumpf, 2007. "The Effect of File Sharing on Record Sales: An Empirical Analysis," Journal of Political Economy, University of Chicago Press, vol. 115(1), pages 1-42.
    7. Justin McCrary, 2002. "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime: Comment," American Economic Review, American Economic Association, vol. 92(4), pages 1236-1243, September.
    8. Racine, Jeffrey, 2001. "On the Nonlinear Predictability of Stock Returns Using Financial and Economic Variables," Journal of Business & Economic Statistics, American Statistical Association, vol. 19(3), pages 380-382, July.
    9. Dewald, William G & Thursby, Jerry G & Anderson, Richard G, 1986. "Replication in Empirical Economics: The Journal of Money, Credit and Banking Project," American Economic Review, American Economic Association, vol. 76(4), pages 587-603, September.
    10. A. W. Phillips, 1958. "The Relation Between Unemployment and the Rate of Change of Money Wage Rates in the United Kingdom, 1861–1957," Economica, London School of Economics and Political Science, vol. 25(100), pages 283-299, November.
    11. B. D. McCullough & H. D. Vinod, 2003. "Verifying the Solution from a Nonlinear Solver: A Case Study," American Economic Review, American Economic Association, vol. 93(3), pages 873-892, June.
    12. Spanos,Aris, 1999. "Probability Theory and Statistical Inference," Cambridge Books, Cambridge University Press, number 9780521424080.
    13. Jesse Rothstein, 2004. "Does Competition Among Public Schools Benefit Students and Taxpayers? A Comment on Hoxby," Working Papers 10, Princeton University, School of Public and International Affairs, Education Research Section..
    14. Bruce Chapman & Yvonne Dunlop & Matthew Gray & Amy Liu & Deborah Mitchell, 2001. "The Impact of Children on the Lifetime Earnings of Australian Women: Evidence from the 1990s," Australian Economic Review, The University of Melbourne, Melbourne Institute of Applied Economic and Social Research, vol. 34(4), pages 373-389, December.
    15. Levitt, Steven D, 1997. "Using Electoral Cycles in Police Hiring to Estimate the Effect of Police on Crime," American Economic Review, American Economic Association, vol. 87(3), pages 270-290, June.
    16. B. D. McCullough & H. D. Vinod, 2004. "Verifying the Solution from a Nonlinear Solver: A Case Study: Reply," American Economic Review, American Economic Association, vol. 94(1), pages 400-406, March.
    17. Jesse Rothstein, 2005. "Does Competition Among Public Schools Benefit Students and Taxpayers? A Comment on Hoxby (2000)," NBER Working Papers 11215, National Bureau of Economic Research, Inc.
    18. Trevor Breusch & Edith Gray, 2004. "New Estimates of Mothers’ Forgone Earnings Using HILDA Data," Australian Journal of Labour Economics (AJLE), Bankwest Curtin Economics Centre (BCEC), Curtin Business School, vol. 7(2), pages 125-150, June.
    19. Caroline M. Hoxby, 2005. "Competition Among Public Schools: A Reply to Rothstein (2004)," NBER Working Papers 11216, National Bureau of Economic Research, Inc.
    20. Wulwick, Nancy J, 1989. "Phillips' Approximate Regression," Oxford Economic Papers, Oxford University Press, vol. 41(1), pages 170-188, January.
    21. Jushan Bai & Pierre Perron, 2003. "Computation and analysis of multiple structural change models," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 18(1), pages 1-22.
    22. B. D. McCullough, 2007. "Got Replicability? The _Journal of Money, Credit and Banking_ Archive," Econ Journal Watch, Econ Journal Watch, vol. 4(3), pages 326-337, September.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Michael A. Clemens, 2017. "The Meaning Of Failed Replications: A Review And Proposal," Journal of Economic Surveys, Wiley Blackwell, vol. 31(1), pages 326-342, February.
    2. Andrew C. Chang & Phillip Li, 2015. "Is Economics Research Replicable? Sixty Published Papers from Thirteen Journals Say \"Usually Not\"," Finance and Economics Discussion Series 2015-83, Board of Governors of the Federal Reserve System (U.S.).
    3. Daniel S. Hamermesh, 2007. "Viewpoint: Replication in economics," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 40(3), pages 715-733, August.
    4. Valérie Orozco & Christophe Bontemps & Elise Maigné & Virginie Piguet & Annie Hofstetter & Anne Lacroix & Fabrice Levert & Jean‐Marc Rousselle, 2020. "How To Make A Pie: Reproducible Research For Empirical Economics And Econometrics," Journal of Economic Surveys, Wiley Blackwell, vol. 34(5), pages 1134-1169, December.
    5. Maren Duvendack & Richard Palmer-Jones, 2013. "Replication of quantitative work in development studies: Experiences and suggestions," Progress in Development Studies, , vol. 13(4), pages 307-322, October.
    6. Robert Chernomas & Ian Hudson, 2013. "Steven Levitt on Abortion and Crime: Old Economics in New Bottles," American Journal of Economics and Sociology, Wiley Blackwell, vol. 72(3), pages 675-700, July.
    7. Hamermesh, Daniel S., 2007. "Replication in Economics," IZA Discussion Papers 2760, Institute of Labor Economics (IZA).
    8. Che, Yi & Xu, Xun, 2015. "The China Syndrome in US: Import Competition, Crime, and Government Transfer," MPRA Paper 68135, University Library of Munich, Germany.
    9. O’Flaherty, Brendan & Sethi, Rajiv, 2015. "Urban Crime," Handbook of Regional and Urban Economics, in: Gilles Duranton & J. V. Henderson & William C. Strange (ed.), Handbook of Regional and Urban Economics, edition 1, volume 5, chapter 0, pages 1519-1621, Elsevier.
    10. Che, Yi & Xu, Xun & Zhang, Yan, 2018. "Chinese import competition, crime, and government transfers in US," Journal of Comparative Economics, Elsevier, vol. 46(2), pages 544-567.
    11. B. D. McCullough, 2007. "Got Replicability? The _Journal of Money, Credit and Banking_ Archive," Econ Journal Watch, Econ Journal Watch, vol. 4(3), pages 326-337, September.
    12. Alejandro Gaviria & Carlos Medina & Jorge Tamayo, 2010. "Assessing the Link between Adolescent Fertility and Urban Crime," Borradores de Economia 6860, Banco de la Republica.
    13. Mueller-Langer, Frank & Andreoli-Versbach, Patrick, 2018. "Open access to research data: Strategic delay and the ambiguous welfare effects of mandatory data disclosure," Information Economics and Policy, Elsevier, vol. 42(C), pages 20-34.
    14. Robert A. Moffitt, 2011. "Report of the Editor: American Economic Review (with Appendix by Philip J. Glandon)," American Economic Review, American Economic Association, vol. 101(3), pages 684-693, May.
    15. Stephen Machin & Olivier Marie, 2011. "Crime And Police Resources: The Street Crime Initiative," Journal of the European Economic Association, European Economic Association, vol. 9(4), pages 678-701, August.
    16. Andreoli-Versbach, Patrick & Mueller-Langer, Frank, 2014. "Open access to data: An ideal professed but not practised," Research Policy, Elsevier, vol. 43(9), pages 1621-1633.
    17. Philip A. Curry & Anindya Sen & George Orlov, 2016. "Crime, apprehension and clearance rates: Panel data evidence from Canadian provinces," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 49(2), pages 481-514, May.
    18. Paolo Buonanno & Giovanni Mastrobuoni, 2012. "Police and Crime: Evidence from Dictated Delays in Centralized Police Hiring," Carlo Alberto Notebooks 244, Collegio Carlo Alberto.
    19. Klick, Jonathan & Tabarrok, Alexander, 2005. "Using Terror Alert Levels to Estimate the Effect of Police on Crime," Journal of Law and Economics, University of Chicago Press, vol. 48(1), pages 267-279, April.
    20. Reyes, Luis Carlos, 2014. "Estimating the Causal Effect of Forced Eradication on Coca Cultivation in Colombian Municipalities," World Development, Elsevier, vol. 61(C), pages 70-84.

    More about this item

    JEL classification:

    • B40 - Schools of Economic Thought and Methodology - - Economic Methodology - - - General
    • C80 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:canjec:v:41:y:2008:i:4:p:1406-1420. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1540-5982 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.