IDEAS home Printed from https://ideas.repec.org/a/wly/amposc/v63y2019i2p491-508.html
   My bibliography  Save this article

Measuring and Explaining Political Sophistication through Textual Complexity

Author

Listed:
  • Kenneth Benoit
  • Kevin Munger
  • Arthur Spirling

Abstract

Political scientists lack domain‐specific measures for the purpose of measuring the sophistication of political communication. We systematically review the shortcomings of existing approaches, before developing a new and better method along with software tools to apply it. We use crowdsourcing to perform thousands of pairwise comparisons of text snippets and incorporate these results into a statistical model of sophistication. This includes previously excluded features such as parts of speech and a measure of word rarity derived from dynamic term frequencies in the Google Books data set. Our technique not only shows which features are appropriate to the political domain and how, but also provides a measure easily applied and rescaled to political texts in a way that facilitates probabilistic comparisons. We reanalyze the State of the Union corpus to demonstrate how conclusions differ when using our improved approach, including the ability to compare complexity as a function of covariates.

Suggested Citation

  • Kenneth Benoit & Kevin Munger & Arthur Spirling, 2019. "Measuring and Explaining Political Sophistication through Textual Complexity," American Journal of Political Science, John Wiley & Sons, vol. 63(2), pages 491-508, April.
    • Handle: RePEc:wly:amposc:v:63:y:2019:i:2:p:491-508
    DOI: 10.1111/ajps.12423
    as

    Download full text from publisher

    File URL: https://doi.org/10.1111/ajps.12423
    Download Restriction: no
    File URL: https://libkey.io/10.1111/ajps.12423?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Turner, Heather & Firth, David, 2012. "Bradley-Terry Models in R: The BradleyTerry2 Package," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i09).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andreas C Goldberg & Lukas Benedikt Hoffmann, 2024. "Peoples’ perspectives on the ‘Future of Europe’ – A comparative study from within and beyond the European Union," European Union Politics, , vol. 25(1), pages 151-172, March.
    2. Amarasinghe, Ashani, 2022. "Diverting domestic turmoil," Journal of Public Economics, Elsevier, vol. 208(C).
    3. Rebecca Cordell & Kristian Skrede Gleditsch & Florian G Kern & Laura Saavedra-Lux, 2020. "Measuring institutional variation across American Indian constitutions using automated content analysis," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(6), pages 777-788, November.
    4. Ferrara, Federico Maria & Angino, Siria, 2022. "Does clarity make central banks more engaging? Lessons from ECB communications," European Journal of Political Economy, Elsevier, vol. 74(C).
    5. Özdemir, Sina & Rauh, Christian, 2022. "A Bird’s Eye View: Supranational EU Actors on Twitter," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 10(1), pages 133-145.
    6. Gloria Gennaro & Elliott Ash, 2022. "Emotion and Reason in Political Language," The Economic Journal, Royal Economic Society, vol. 132(643), pages 1037-1059.
    7. Bernhard Ganglmair & Julia Krämer & Jacopo Gambato, 2024. "Regulatory Compliance with Limited Enforceability: Evidence from Privacy Policies," CRC TR 224 Discussion Paper Series crctr224_2024_547v2, University of Bonn and University of Mannheim, Germany, revised Sep 2024.
    8. Vyshnevskyi, Iegor & Jombo, Wytone & Sohn, Wook, 2024. "The clarity of monetary policy communication and financial market volatility in developing economies," Emerging Markets Review, Elsevier, vol. 59(C).
    9. Alexander Baturo & Julia Gray, 2024. "Leaders in the United Nations General Assembly: Revitalization or politicization?," The Review of International Organizations, Springer, vol. 19(4), pages 721-752, October.
    10. Hamza Bennani & Matthias Neuenkirch, 2024. "Too complex to digest? Federal tax bills and their processing in US financial markets," International Tax and Public Finance, Springer;International Institute of Public Finance, vol. 31(5), pages 1179-1203, October.
    11. Roberta Rocca & Katharina Lawall & Manos Tsakiris & Laura Cram, 2024. "Communicating Europe: a computational analysis of the evolution of the European Commission’s communication on Twitter," Journal of Computational Social Science, Springer, vol. 7(2), pages 1223-1274, October.
    12. Rauh, Christian, 2022. "Clear messages to the European public? The language of European Commission press releases 1985–2020," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, issue Latest Ar, pages 1-19.
    13. Adam M. Samaha & Michael Heise & Gregory C. Sisk, 2020. "Inputs and Outputs on Appeal: An Empirical Study of Briefs, Big Law, and Case Complexity," Journal of Empirical Legal Studies, John Wiley & Sons, vol. 17(3), pages 519-555, September.
    14. Sivaram Cheruvu, 2019. "How do institutional constraints affect judicial decision-making? The European Court of Justice’s French language mandate," European Union Politics, , vol. 20(4), pages 562-583, December.
    15. Sina Özdemir & Christian Rauh, 2022. "A Bird’s Eye View: Supranational EU Actors on Twitter," Politics and Governance, Cogitatio Press, vol. 10(1), pages 133-145.
    16. Eddy Cardinaels & Christoph Feichter, 2021. "Forced Rating Systems from Employee and Supervisor Perspectives," Journal of Accounting Research, Wiley Blackwell, vol. 59(5), pages 1573-1607, December.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Cristiano Varin & Manuela Cattelan & David Firth, 2016. "Statistical modelling of citation exchange between statistics journals," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 179(1), pages 1-63, January.
    2. Kovalchik Stephanie Ann, 2016. "Searching for the GOAT of tennis win prediction," Journal of Quantitative Analysis in Sports, De Gruyter, vol. 12(3), pages 127-138, September.
    3. Francesca Giambona & Mariano Porcu & Isabella Sulis, 2017. "Students Mobility: Assessing the Determinants of Attractiveness Across Competing Territorial Areas," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 133(3), pages 1105-1132, September.
    4. Anna Gottard & Giorgio Calzolari, 2014. "Alternative estimating procedures for multiple membership logit models with mixed effects: indirect inference and data cloning," Econometrics Working Papers Archive 2014_07, Universita' degli Studi di Firenze, Dipartimento di Statistica, Informatica, Applicazioni "G. Parenti".
    5. Vicente Rodríguez Montequín & Joaquín Manuel Villanueva Balsera & Marina Díaz Piloñeta & César Álvarez Pérez, 2020. "A Bradley-Terry Model-Based Approach to Prioritize the Balance Scorecard Driving Factors: The Case Study of a Financial Software Factory," Mathematics, MDPI, vol. 8(2), pages 1-15, February.
    6. Manuela Cattelan & Cristiano Varin, 2013. "Hybrid Pairwise Likelihood Analysis of Animal Behavior Experiments," Biometrics, The International Biometric Society, vol. 69(4), pages 1002-1011, December.
    7. Wickelmaier, Florian & Strobl, Carolin & Zeileis, Achim, 2012. "Psychoco: Psychometric Computing in R," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 48(i01).
    8. Collingwood, James A.P. & Wright, Michael & Brooks, Roger J, 2022. "Evaluating the effectiveness of different player rating systems in predicting the results of professional snooker matches," European Journal of Operational Research, Elsevier, vol. 296(3), pages 1025-1035.
    9. Hankin, Robin K.S., 2020. "A generalization of the Bradley–Terry model for draws in chess with an application to collusion," Journal of Economic Behavior & Organization, Elsevier, vol. 180(C), pages 325-333.
    10. Mai Phuong Nguyen & Philippe Vaast & Tim Pagella & Fergus Sinclair, 2020. "Local Knowledge about Ecosystem Services Provided by Trees in Coffee Agroforestry Practices in Northwest Vietnam," Land, MDPI, vol. 9(12), pages 1-27, December.
    11. Weichen Wu & Nynke Niezink & Brian Junker, 2022. "A diagnostic framework for the Bradley–Terry model," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 185(S2), pages 461-484, December.
    12. Gunther Schauberger & Andreas Groll & Gerhard Tutz, 2018. "Analysis of the importance of on-field covariates in the German Bundesliga," Journal of Applied Statistics, Taylor & Francis Journals, vol. 45(9), pages 1561-1578, July.
    13. Alwyn Lim & Shawn Pope, 2022. "What drives companies to do good? A “universal” ordering of corporate social responsibility motivations," Corporate Social Responsibility and Environmental Management, John Wiley & Sons, vol. 29(1), pages 233-255, January.
    14. Rowland G. Seymour & David Sirl & Simon P. Preston & Ian L. Dryden & Madeleine J. A. Ellis & Bertrand Perrat & James Goulding, 2022. "The Bayesian Spatial Bradley–Terry model: Urban deprivation modelling in Tanzania," Journal of the Royal Statistical Society Series C, Royal Statistical Society, vol. 71(2), pages 288-308, March.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:wly:amposc:v:63:y:2019:i:2:p:491-508. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Wiley Content Delivery (email available below). General contact details of provider: https://doi.org/10.1111/(ISSN)1540-5907 .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.