IDEAS home Printed from https://ideas.repec.org/a/eee/irlaec/v62y2020ics0144818819302571.html
   My bibliography  Save this article

Measuring discourse by algorithm

Author

Listed:
  • Caspi, Aviv
  • Stiglitz, Edward H.

Abstract

Scholars increasingly use machine learning techniques such as Latent Dirichlet Allocation (LDA) to reduce the dimensionality of textual data and to study discourse in collective bodies. However, measures of discourse based on algorithmic results typically have no intuitive meaning or obvious relationship to humanly observed discourse. Such measures of discourse must be carefully validated before relied on and interpreted. We examine several common measures of discourse based on algorithmic results, and propose a number of ways to validate them in the setting of Federal Open Market Committee meetings. We also suggest that validation techniques may be used as a principled approach to model selection and parameterization.

Suggested Citation

  • Caspi, Aviv & Stiglitz, Edward H., 2020. "Measuring discourse by algorithm," International Review of Law and Economics, Elsevier, vol. 62(C).
    • Handle: RePEc:eee:irlaec:v:62:y:2020:i:c:s0144818819302571
    DOI: 10.1016/j.irle.2019.105863
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0144818819302571
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.irle.2019.105863?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Stephen Hansen & Michael McMahon & Andrea Prat, 2018. "Transparency and Deliberation Within the FOMC: A Computational Linguistics Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 801-870.
    2. Margaret E. Roberts & Brandon M. Stewart & Edoardo M. Airoldi, 2016. "A Model of Text for Experimentation in the Social Sciences," Journal of the American Statistical Association, Taylor & Francis Journals, vol. 111(515), pages 988-1003, July.
    3. Bengt Holmström, 1999. "Managerial Incentive Problems: A Dynamic Perspective," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 66(1), pages 169-182.
    4. Kevin M. Quinn & Burt L. Monroe & Michael Colaresi & Michael H. Crespin & Dragomir R. Radev, 2010. "How to Analyze Political Attention with Minimal Assumptions and Costs," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 209-228, January.
    5. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    6. Bauke Visser & Otto H. Swank, 2007. "On Committees of Experts," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 122(1), pages 337-372.
    7. Parthasarathy, Ramya & Rao, Vijayendra & Palaniswamy, Nethra, 2019. "Deliberative Democracy in an Unequal World: A Text-As-Data Study of South India’s Village Assemblies," American Political Science Review, Cambridge University Press, vol. 113(3), pages 623-640, August.
    8. Ellen E. Meade, 2005. "The FOMC: preferences, voting, and consensus," Review, Federal Reserve Bank of St. Louis, vol. 87(Mar), pages 93-101.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Chin, Jason & Zeiler, Kathryn, 2021. "Replicability in Empirical Legal Research," LawArXiv 2b5k4, Center for Open Science.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Mohamed M. Mostafa, 2023. "A one-hundred-year structural topic modeling analysis of the knowledge structure of international management research," Quality & Quantity: International Journal of Methodology, Springer, vol. 57(4), pages 3905-3935, August.
    2. David Bholat & Stephen Hans & Pedro Santos & Cheryl Schonhardt-Bailey, 2015. "Text mining for central banks," Handbooks, Centre for Central Banking Studies, Bank of England, number 33, April.
    3. Dehler-Holland, Joris & Schumacher, Kira & Fichtner, Wolf, 2021. "Topic Modeling Uncovers Shifts in Media Framing of the German Renewable Energy Act," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 2(1).
    4. Ehrmann, Michael & Gnan, Phillipp & Rieder, Kilian, 2023. "Central Bank Communication by ??? The Economics of Public Policy Leaks," CEPR Discussion Papers 18152, C.E.P.R. Discussion Papers.
    5. Dehler-Holland, Joris & Okoh, Marvin & Keles, Dogan, 2022. "Assessing technology legitimacy with topic models and sentiment analysis – The case of wind power in Germany," Technological Forecasting and Social Change, Elsevier, vol. 175(C).
    6. Lino Wehrheim, 2019. "Economic history goes digital: topic modeling the Journal of Economic History," Cliometrica, Springer;Cliometric Society (Association Francaise de Cliométrie), vol. 13(1), pages 83-125, January.
    7. Peter Grajzl & Peter Murrell, 2021. "Characterizing a legal–intellectual culture: Bacon, Coke, and seventeenth-century England," Cliometrica, Journal of Historical Economics and Econometric History, Association Française de Cliométrie (AFC), vol. 15(1), pages 43-88, January.
    8. Peter Grajzl & Cindy Irby, 2019. "Reflections on study abroad: a computational linguistics approach," Journal of Computational Social Science, Springer, vol. 2(2), pages 151-181, July.
    9. Ashwin,Julian & Rao,Vijayendra & Biradavolu,Monica Rao & Chhabra,Aditya & Haque,Arshia & Khan,Afsana Iffat & Krishnan,Nandini, 2022. "A Method to Scale-Up Interpretative Qualitative Analysis, with an Application toAspirations in Cox’s Bazaar, Bangladesh," Policy Research Working Paper Series 10046, The World Bank.
    10. Matthew Gentzkow & Bryan T. Kelly & Matt Taddy, 2017. "Text as Data," NBER Working Papers 23276, National Bureau of Economic Research, Inc.
    11. Lino Wehrheim, 2017. "Economic History Goes Digital: Topic Modeling the Journal of Economic History," Working Papers 177, Bavarian Graduate Program in Economics (BGPE).
    12. Mourtgos, Scott M. & Adams, Ian T., 2019. "The rhetoric of de-policing: Evaluating open-ended survey responses from police officers with machine learning-based structural topic modeling," Journal of Criminal Justice, Elsevier, vol. 64(C), pages 1-1.
    13. Valentina A. Bali & Devin Higgins, 2023. "More than Meets the Eye? Using Text Analytic Techniques to Unpack School Mission Statements," SAGE Open, , vol. 13(4), pages 21582440231, December.
    14. Mueller, Hannes & Rauh, Christopher, 2018. "Reading Between the Lines: Prediction of Political Violence Using Newspaper Text," American Political Science Review, Cambridge University Press, vol. 112(2), pages 358-375, May.
    15. Sandra Wankmüller, 2023. "A comparison of approaches for imbalanced classification problems in the context of retrieving relevant documents for an analysis," Journal of Computational Social Science, Springer, vol. 6(1), pages 91-163, April.
    16. McCannon, Bryan & Zhou, Yang & Hall, Joshua, 2021. "Measuring a Contract’s Breadth: A Text Analysis," Working Papers 11013, George Mason University, Mercatus Center.
    17. Grajzl, Peter & Murrell, Peter, 2021. "A machine-learning history of English caselaw and legal ideas prior to the Industrial Revolution I: generating and interpreting the estimates," Journal of Institutional Economics, Cambridge University Press, vol. 17(1), pages 1-19, February.
    18. Stephen Hansen & Michael McMahon & Andrea Prat, 2018. "Transparency and Deliberation Within the FOMC: A Computational Linguistics Approach," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(2), pages 801-870.
    19. Marcel Fratzscher & Tobias Heidland & Lukas Menkhoff & Lucio Sarno & Maik Schmeling, 2023. "Foreign Exchange Intervention: A New Database," IMF Economic Review, Palgrave Macmillan;International Monetary Fund, vol. 71(4), pages 852-884, December.
    20. Arthur Dyevre & Nicolas Lampach, 2021. "Issue attention on international courts: Evidence from the European Court of Justice," The Review of International Organizations, Springer, vol. 16(4), pages 793-815, October.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:irlaec:v:62:y:2020:i:c:s0144818819302571. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/irle .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.