IDEAS home Printed from https://ideas.repec.org/a/aea/jeclit/v58y2020i1p3-52.html
   My bibliography  Save this article

Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship

Author

Listed:
  • Josh Angrist
  • Pierre Azoulay
  • Glenn Ellison
  • Ryan Hill
  • Susan Feng Lu

Abstract

Does academic economic research produce material of general scientific value, or do academic economists write only for peers? Is economics scholarship uniquely insular? We address these questions by quantifying interactions between economics and other disciplines. Changes in the influence of economic scholarship are measured here by the frequency with which other disciplines cite papers in economics journals. We document a clear rise in the extramural influence of economic research, while also showing that economics is increasingly likely to reference other social sciences. A breakdown of extramural citations by economics fields shows broad field influence. Differentiating between theoretical and empirical papers classified using machine learning, we see that much of the rise in economics' extramural influence reflects growth in citations to empirical work. This growth parallels an increase in the share of empirical cites within economics. At the same time, some disciplines that primarily cite economic theory have also recently increased citations of economics scholarship.

Suggested Citation

  • Josh Angrist & Pierre Azoulay & Glenn Ellison & Ryan Hill & Susan Feng Lu, 2020. "Inside Job or Deep Impact? Extramural Citations and the Influence of Economic Scholarship," Journal of Economic Literature, American Economic Association, vol. 58(1), pages 3-52, March.
    • Handle: RePEc:aea:jeclit:v:58:y:2020:i:1:p:3-52
    DOI: 10.1257/jel.20181508
    as

    Download full text from publisher

    File URL: https://www.aeaweb.org/doi/10.1257/jel.20181508
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/jel.20181508.appx
    Download Restriction: no
    File URL: https://www.aeaweb.org/doi/10.1257/jel.20181508.ds
    Download Restriction: no
    File URL: https://libkey.io/10.1257/jel.20181508?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Romer, Paul M, 1986. "Increasing Returns and Long-run Growth," Journal of Political Economy, University of Chicago Press, vol. 94(5), pages 1002-1037, October.
    2. Joshua D. Angrist & Jörn-Steffen Pischke, 2010. "The Credibility Revolution in Empirical Economics: How Better Research Design Is Taking the Con out of Econometrics," Journal of Economic Perspectives, American Economic Association, vol. 24(2), pages 3-30, Spring.
    3. Matthew T. Panhans & John D. Singleton, 2015. "The Empirical Economist's Toolkit: From Models to Methods," Center for the History of Political Economy Working Paper Series 2015-3, Center for the History of Political Economy.
    4. Hall, Robert E, 1978. "Stochastic Implications of the Life Cycle-Permanent Income Hypothesis: Theory and Evidence," Journal of Political Economy, University of Chicago Press, vol. 86(6), pages 971-987, December.
    5. Hirotaka Miura, 2012. "Stata graph library for network analysis," Stata Journal, StataCorp LP, vol. 12(1), pages 94-129, March.
    6. Hausman, Jerry, 2015. "Specification tests in econometrics," Applied Econometrics, Russian Presidential Academy of National Economy and Public Administration (RANEPA), vol. 38(2), pages 112-134.
    7. Glenn Ellison, 2013. "How Does the Market Use Citation Data? The Hirsch Index in Economics," American Economic Journal: Applied Economics, American Economic Association, vol. 5(3), pages 63-90, July.
    8. Beatrice Cherrier, 2017. "Classifying Economics: A History of the JEL Codes," Journal of Economic Literature, American Economic Association, vol. 55(2), pages 545-579, June.
    9. Imbens, Guido W & Angrist, Joshua D, 1994. "Identification and Estimation of Local Average Treatment Effects," Econometrica, Econometric Society, vol. 62(2), pages 467-475, March.
    10. Axel Ockenfels & Gary E. Bolton, 2000. "ERC: A Theory of Equity, Reciprocity, and Competition," American Economic Review, American Economic Association, vol. 90(1), pages 166-193, March.
    11. Lawrence J. Christiano & Martin Eichenbaum & Charles L. Evans, 2005. "Nominal Rigidities and the Dynamic Effects of a Shock to Monetary Policy," Journal of Political Economy, University of Chicago Press, vol. 113(1), pages 1-45, February.
    12. Daniel S. Hamermesh, 2013. "Six Decades of Top Economics Publishing: Who and How?," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 162-172, March.
    13. Marianne Bertrand & Esther Duflo & Sendhil Mullainathan, 2004. "How Much Should We Trust Differences-In-Differences Estimates?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 119(1), pages 249-275.
    14. Daniel Kahneman & Amos Tversky, 2013. "Prospect Theory: An Analysis of Decision Under Risk," World Scientific Book Chapters, in: Leonard C MacLean & William T Ziemba (ed.), HANDBOOK OF THE FUNDAMENTALS OF FINANCIAL DECISION MAKING Part I, chapter 6, pages 99-127, World Scientific Publishing Co. Pte. Ltd..
    15. Robert E. Hall & Charles I. Jones, 1999. "Why do Some Countries Produce So Much More Output Per Worker than Others?," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 114(1), pages 83-116.
    16. Imbens, Guido W. & Lemieux, Thomas, 2008. "Regression discontinuity designs: A guide to practice," Journal of Econometrics, Elsevier, vol. 142(2), pages 615-635, February.
    17. Urs Fischbacher, 2007. "z-Tree: Zurich toolbox for ready-made economic experiments," Experimental Economics, Springer;Economic Science Association, vol. 10(2), pages 171-178, June.
    18. Rik Pieters & Hans Baumgartner, 2002. "Who Talks to Whom? Intra- and Interdisciplinary Communication of Economics Journals," Journal of Economic Literature, American Economic Association, vol. 40(2), pages 483-509, June.
    19. Shaun McRae, 2015. "Infrastructure Quality and the Subsidy Trap," American Economic Review, American Economic Association, vol. 105(1), pages 35-66, January.
    20. White, Halbert, 1980. "A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity," Econometrica, Econometric Society, vol. 48(4), pages 817-838, May.
    21. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    22. David Card & Stefano DellaVigna, 2013. "Nine Facts about Top Journals in Economics," Journal of Economic Literature, American Economic Association, vol. 51(1), pages 144-161, March.
    23. Michael A. Kelly & Stephen Bruestle, 2011. "Trend Of Subjects Published In Economics Journals 1969–2007," Economic Inquiry, Western Economic Association International, vol. 49(3), pages 658-673, July.
    24. Loet Leydesdorff, 2004. "Top-down decomposition of the Journal Citation Reportof the Social Science Citation Index: Graph- and factor-analytical approaches," Scientometrics, Springer;Akadémiai Kiadó, vol. 60(2), pages 159-180, June.
    25. Hausman,Daniel M., 1992. "The Inexact and Separate Science of Economics," Cambridge Books, Cambridge University Press, number 9780521415019, December.
    26. Ofer H. Azar, 2009. "The Influence Of Economics Articles On Business Research: Analysis Of Journals And Time Trends," Journal of Industrial Economics, Wiley Blackwell, vol. 57(4), pages 851-869, December.
    27. David Laibson, 1997. "Golden Eggs and Hyperbolic Discounting," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 112(2), pages 443-478.
    28. repec:wly:soecon:v:82:2:y:2015:p:430-452 is not listed on IDEAS
    29. George A. Akerlof, 1970. "The Market for "Lemons": Quality Uncertainty and the Market Mechanism," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 84(3), pages 488-500.
    30. Hausman,Daniel M., 1992. "The Inexact and Separate Science of Economics," Cambridge Books, Cambridge University Press, number 9780521425230, December.
    31. Kydland, Finn E & Prescott, Edward C, 1977. "Rules Rather Than Discretion: The Inconsistency of Optimal Plans," Journal of Political Economy, University of Chicago Press, vol. 85(3), pages 473-491, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Joshua Angrist & Pierre Azoulay & Glenn Ellison & Ryan Hill & Susan Feng Lu, 2017. "Inside Job or Deep Impact? Using Extramural Citations to Assess Economic Scholarship," NBER Working Papers 23698, National Bureau of Economic Research, Inc.
    2. Konstantinos Metaxoglou, 2021. "Canadian Journal of Economics: A historic overview," Canadian Journal of Economics/Revue canadienne d'économique, John Wiley & Sons, vol. 54(3), pages 1418-1453, November.
    3. Joshua D. Angrist & Jörn-Steffen Pischke, 2017. "Undergraduate Econometrics Instruction: Through Our Classes, Darkly," Journal of Economic Perspectives, American Economic Association, vol. 31(2), pages 125-144, Spring.
    4. Marion Fourcade & Etienne Ollion & Yann Algan, 2015. "La superioridad de los economistas," Revista de Economía Institucional, Universidad Externado de Colombia - Facultad de Economía, vol. 17(33), pages 13-43, July-Dece.
    5. Michel De Vroey & Luca Pensieroso, 2021. "Grounded in Methodology, Certified by Journals: The Rise and Evolution of a Mainstream in Economics," LIDAM Discussion Papers IRES 2021015, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    6. Aleksey Oshchepkov & Anna Shirokanova, 2020. "Multilevel Modeling For Economists: Why, When And How," HSE Working papers WP BRP 233/EC/2020, National Research University Higher School of Economics.
    7. Ivan Moscati, 2022. "Behavioral and heuristic models are as-if models too — and that’s ok," BAFFI CAREFIN Working Papers 22177, BAFFI CAREFIN, Centre for Applied Research on International Markets Banking Finance and Regulation, Universita' Bocconi, Milano, Italy.
    8. Syed Hasan & Robert Breunig, 2021. "Article length and citation outcomes," Scientometrics, Springer;Akadémiai Kiadó, vol. 126(9), pages 7583-7608, September.
    9. Mark J. McCabe & Frank Mueller-Langer, 2019. "Does Data Disclosure Increase Citations? Empirical Evidence from a Natural Experiment in Leading Economics Journals," JRC Working Papers on Digital Economy 2019-02, Joint Research Centre.
    10. Adrian C. Darnell, 1994. "A Dictionary Of Econometrics," Books, Edward Elgar Publishing, number 118.
    11. Bogliacino, Francesco & Codagnone, Cristiano, 2021. "Microfoundations, behaviour, and evolution: Evidence from experiments," Structural Change and Economic Dynamics, Elsevier, vol. 56(C), pages 372-385.
    12. Daniel S. Hamermesh, 2018. "Citations in Economics: Measurement, Uses, and Impacts," Journal of Economic Literature, American Economic Association, vol. 56(1), pages 115-156, March.
    13. M. Fourcade & E. Ollion & Y. Algan, 2015. "The Superiority of Economists," Voprosy Ekonomiki, NP Voprosy Ekonomiki, issue 7.
    14. José Edwards & Yann Giraud & Christophe Schinckus, 2018. "A quantitative turn in the historiography of economics?," Journal of Economic Methodology, Taylor & Francis Journals, vol. 25(4), pages 283-290, October.
    15. Robin Maialeh, 2019. "Generalization of results and neoclassical rationality: unresolved controversies of behavioural economics methodology," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(4), pages 1743-1761, July.
    16. Püttmann, Vitus & Thomsen, Stephan L. & Trunzer, Johannes, 2020. "Zur Relevanz von Ausstattungsunterschieden für Forschungsleistungsvergleiche: Ein Diskussionsbeitrag für die Wirtschaftswissenschaften in Deutschland," Hannover Economic Papers (HEP) dp-679, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Mar 2021.
    17. Andrei Dubovik & Clemens Fiedler & Alexei Parakhonyak, 2022. "Temporal Patterns in Economics Research," CPB Discussion Paper 440, CPB Netherlands Bureau for Economic Policy Analysis.
    18. Michel De Vroey & Luca Pensieroso, 2016. "The Rise of a Mainstream in Economics," LIDAM Discussion Papers IRES 2016026, Université catholique de Louvain, Institut de Recherches Economiques et Sociales (IRES).
    19. María Victoria Anauati & Sebastian Galiani & Ramiro H. Gálvez, 2020. "Differences In Citation Patterns Across Journal Tiers: The Case Of Economics," Economic Inquiry, Western Economic Association International, vol. 58(3), pages 1217-1232, July.
    20. Jan Toporowski, 2013. "The Elgar Companion to Hyman Minsky," Review of Political Economy, Taylor & Francis Journals, vol. 25(1), pages 175-177, January.

    More about this item

    JEL classification:

    • A11 - General Economics and Teaching - - General Economics - - - Role of Economics; Role of Economists
    • A14 - General Economics and Teaching - - General Economics - - - Sociology of Economics

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:aea:jeclit:v:58:y:2020:i:1:p:3-52. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Michael P. Albert (email available below). General contact details of provider: https://edirc.repec.org/data/aeaaaea.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.