IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1809.00741.html
   My bibliography  Save this paper

"Read My Lips": Using Automatic Text Analysis to Classify Politicians by Party and Ideology

Author

Listed:
  • Eitan Sapiro-Gheiler

Abstract

The increasing digitization of political speech has opened the door to studying a new dimension of political behavior using text analysis. This work investigates the value of word-level statistical data from the US Congressional Record--which contains the full text of all speeches made in the US Congress--for studying the ideological positions and behavior of senators. Applying machine learning techniques, we use this data to automatically classify senators according to party, obtaining accuracy in the 70-95% range depending on the specific method used. We also show that using text to predict DW-NOMINATE scores, a common proxy for ideology, does not improve upon these already-successful results. This classification deteriorates when applied to text from sessions of Congress that are four or more years removed from the training set, pointing to a need on the part of voters to dynamically update the heuristics they use to evaluate party based on political speech. Text-based predictions are less accurate than those based on voting behavior, supporting the theory that roll-call votes represent greater commitment on the part of politicians and are thus a more accurate reflection of their ideological preferences. However, the overall success of the machine learning approaches studied here demonstrates that political speeches are highly predictive of partisan affiliation. In addition to these findings, this work also introduces the computational tools and methods relevant to the use of political speech data.

Suggested Citation

  • Eitan Sapiro-Gheiler, 2018. ""Read My Lips": Using Automatic Text Analysis to Classify Politicians by Party and Ideology," Papers 1809.00741, arXiv.org.
  • Handle: RePEc:arx:papers:1809.00741
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1809.00741
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. François Pétry & Benoît Collette, 2009. "Measuring How Political Parties Keep Their Promises: A Positive Perspective from Political Science," Studies in Public Choice, in: Louis M. Imbeau (ed.), Do They Walk Like They Talk?, chapter 0, pages 65-80, Springer.
    2. Lauderdale, Benjamin E. & Herzog, Alexander, 2016. "Measuring Political Positions from Legislative Speech," Political Analysis, Cambridge University Press, vol. 24(3), pages 374-394, July.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Sami Diaf & Jörg Döpke & Ulrich Fritsche & Ida Rockenbach, 2020. "Sharks and minnows in a shoal of words: Measuring latent ideological positions of German economic research institutes based on text mining techniques," Macroeconomics and Finance Series 202001, University of Hamburg, Department of Socioeconomics.
    2. Walkowitz, Gari & Weiss, Arne R., 2017. "“Read my lips! (but only if I was elected)!” Experimental evidence on the effects of electoral competition on promises, shirking and trust," Journal of Economic Behavior & Organization, Elsevier, vol. 142(C), pages 348-367.
    3. Gari Walkowitz & Arne R. Weiss, 2014. ""Read my Lips!" Experimental Evidence on the Effects of Electoral Competition on Shirking and Trust," Cologne Graduate School Working Paper Series 05-07, Cologne Graduate School in Management, Economics and Social Sciences.
    4. Mónica D. Oliveira & Inês Mataloto & Panos Kanavos, 2019. "Multi-criteria decision analysis for health technology assessment: addressing methodological challenges to improve the state of the art," The European Journal of Health Economics, Springer;Deutsche Gesellschaft für Gesundheitsökonomie (DGGÖ), vol. 20(6), pages 891-918, August.
    5. Diaf, Sami & Döpke, Jörg & Fritsche, Ulrich & Rockenbach, Ida, 2022. "Sharks and minnows in a shoal of words: Measuring latent ideological positions based on text mining techniques," European Journal of Political Economy, Elsevier, vol. 75(C).
    6. Christopher Wratil & Sara B Hobolt, 2019. "Public deliberations in the Council of the European Union: Introducing and validating DICEU," European Union Politics, , vol. 20(3), pages 511-531, September.
    7. Caroline Bhattacharya, 2020. "Gatekeeping the Plenary Floor: Discourse Network Analysis as a Novel Approach to Party Control," Politics and Governance, Cogitatio Press, vol. 8(2), pages 229-242.
    8. Millner, Antony & Ollivier, Hélène & Simon, Leo, 2020. "Confirmation bias and signaling in Downsian elections," Journal of Public Economics, Elsevier, vol. 185(C).
    9. Chitralekha Basu & Carles Boix & Sonia Giurumescu & Paulo Serôdio, 2022. "Democratizing from Within: British Elites and the Expansion of the Franchise," ECONtribute Discussion Papers Series 139, University of Bonn and University of Cologne, Germany.
    10. Ash, Elliott & Gauthier, Germain & Widmer, Philine, 2024. "Relatio: Text Semantics Capture Political and Economic Narratives," Political Analysis, Cambridge University Press, vol. 32(1), pages 115-132, January.
    11. Munnes, Stefan & Harsch, Corinna & Knobloch, Marcel & Vogel, Johannes S. & Hipp, Lena & Schilling, Erik, 2022. "Examining Sentiment in Complex Texts. A Comparison of Different Computational Approaches," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 5, pages 1-1.
    12. Hausladen, Carina I. & Schubert, Marcel H. & Ash, Elliott, 2020. "Text classification of ideological direction in judicial opinions," International Review of Law and Economics, Elsevier, vol. 62(C).
    13. Rong, Rong & Barton, Jared, 2021. "I’ll be there: Promises in the field," Organizational Behavior and Human Decision Processes, Elsevier, vol. 166(C), pages 20-26.
    14. Salla Simola & Jeremias Nieminen & Janne Tukiainen, 2023. "A century of partisanship in Finnish political speech," Discussion Papers 160, Aboa Centre for Economics.
    15. Jon H. Fiva & Oda Nedregård & Henning Øien, 2021. "Polarization in Parliamentary Speech," CESifo Working Paper Series 8818, CESifo.
    16. Miriam Barnum & James Lo, 2020. "Is the NPT unraveling? Evidence from text analysis of review conference statements," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(6), pages 740-751, November.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1809.00741. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.