IDEAS home Printed from https://ideas.repec.org/a/spr/qualqt/v51y2017i6d10.1007_s11135-016-0412-4.html
   My bibliography  Save this article

Sentiment analysis of political communication: combining a dictionary approach with crowdcoding

Author

Listed:
  • Martin Haselmayer

    (University of Vienna)

  • Marcelo Jenny

    (University of Vienna)

Abstract

Sentiment is important in studies of news values, public opinion, negative campaigning or political polarization and an explosive expansion of digital textual data and fast progress in automated text analysis provide vast opportunities for innovative social science research. Unfortunately, tools currently available for automated sentiment analysis are mostly restricted to English texts and require considerable contextual adaption to produce valid results. We present a procedure for collecting fine-grained sentiment scores through crowdcoding to build a negative sentiment dictionary in a language and for a domain of choice. The dictionary enables the analysis of large text corpora that resource-intensive hand-coding struggles to cope with. We calculate the tonality of sentences from dictionary words and we validate these estimates with results from manual coding. The results show that the crowdbased dictionary provides efficient and valid measurement of sentiment. Empirical examples illustrate its use by analyzing the tonality of party statements and media reports.

Suggested Citation

  • Martin Haselmayer & Marcelo Jenny, 2017. "Sentiment analysis of political communication: combining a dictionary approach with crowdcoding," Quality & Quantity: International Journal of Methodology, Springer, vol. 51(6), pages 2623-2646, November.
    • Handle: RePEc:spr:qualqt:v:51:y:2017:i:6:d:10.1007_s11135-016-0412-4
    DOI: 10.1007/s11135-016-0412-4
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11135-016-0412-4
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11135-016-0412-4?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Benoit, Kenneth & Conway, Drew & Lauderdale, Benjamin E. & Laver, Michael & Mikhaylov, Slava, 2016. "Crowd-sourced Text Analysis: Reproducible and Agile Production of Political Data," American Political Science Review, Cambridge University Press, vol. 110(2), pages 278-295, May.
    2. Mike Thelwall & Kevan Buckley, 2013. "Topic-based sentiment analysis for the social web: The role of mood and issue-related words," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 64(8), pages 1608-1617, August.
    3. Mutz, Diana C. & Reeves, Byron, 2005. "The New Videomalaise: Effects of Televised Incivility on Political Trust," American Political Science Review, Cambridge University Press, vol. 99(1), pages 1-15, February.
    4. Mike Thelwall & Kevan Buckley & Georgios Paltoglou & Di Cai & Arvid Kappas, 2010. "Sentiment strength detection in short informal text," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(12), pages 2544-2558, December.
    5. Mike Thelwall & Kevan Buckley & Georgios Paltoglou & Di Cai & Arvid Kappas, 2010. "Sentiment strength detection in short informal text," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(12), pages 2544-2558, December.
    6. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    7. Lowe, Will & Benoit, Kenneth, 2013. "Validating Estimates of Latent Traits from Textual Data Using Human Judgment as a Benchmark," Political Analysis, Cambridge University Press, vol. 21(3), pages 298-313, July.
    8. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    9. Monroe, Burt L. & Colaresi, Michael P. & Quinn, Kevin M., 2008. "Fightin' Words: Lexical Feature Selection and Evaluation for Identifying the Content of Political Conflict," Political Analysis, Cambridge University Press, vol. 16(4), pages 372-403.
    10. Jonathan B. Slapin & Sven‐Oliver Proksch, 2008. "A Scaling Model for Estimating Time‐Series Party Positions from Texts," American Journal of Political Science, John Wiley & Sons, vol. 52(3), pages 705-722, July.
    11. Laver, Michael & Benoit, Kenneth & Garry, John, 2003. "Extracting Policy Positions from Political Texts Using Words as Data," American Political Science Review, Cambridge University Press, vol. 97(2), pages 311-331, May.
    12. Daniel J. Hopkins & Gary King, 2010. "A Method of Automated Nonparametric Content Analysis for Social Science," American Journal of Political Science, John Wiley & Sons, vol. 54(1), pages 229-247, January.
    13. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    14. Berinsky, Adam J. & Huber, Gregory A. & Lenz, Gabriel S., 2012. "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk," Political Analysis, Cambridge University Press, vol. 20(3), pages 351-368, July.
    15. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2012. "Sentiment strength detection for the social web," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(1), pages 163-173, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Shesen Guo & Ganzhou Zhang, 2020. "Using Machine Learning for Analyzing Sentiment Orientations Toward Eight Countries," SAGE Open, , vol. 10(3), pages 21582440209, August.
    2. Julia Wolfinger & Lars P. Feld & Ekkehard A. Köhler & Tobias Thomas, 2018. "57 Channels (And Nothin On) - Does TV-News on the Eurozone Affect Government Bond Yield Spreads?," CESifo Working Paper Series 7437, CESifo.
    3. Juha Koljonen & Emily Öhman & Pertti Ahonen & Mikko Mattila, 2022. "Strategic sentiments and emotions in post-Second World War party manifestos in Finland," Journal of Computational Social Science, Springer, vol. 5(2), pages 1529-1554, November.
    4. Eyal Eckhaus & Zachary Sheaffer, 2018. "Managerial hubris detection: the case of Enron," Risk Management, Palgrave Macmillan, vol. 20(4), pages 304-325, November.
    5. Katja Pietrzyck & Nora Berke & Vanessa Wendel & Julia Steinhoff-Wagner & Sebastian Jarzębowski & Brigitte Petersen, 2021. "Understanding the Importance of International Quality Standards Regarding Global Trade in Food and Agricultural Products: Analysis of the German Media," Agriculture, MDPI, vol. 11(4), pages 1-20, April.
    6. Hugo Oriola & Matthieu Picault, 2023. "Opportunistic Political Central Bank Coverage: Does media coverage of ECB's Monetary Policy Impacts German Political Parties' Popularity?," EconomiX Working Papers 2023-30, University of Paris Nanterre, EconomiX.
    7. Ralf Dewenter & Uwe Dulleck & Tobias Thomas, 2020. "Does the 4th estate deliver? The Political Coverage Index and its application to media capture," Constitutional Political Economy, Springer, vol. 31(3), pages 292-328, September.
    8. Hirsch, Patrick & Feld, Lars P. & Köhler, Ekkehard A. & Thomas, Tobias, 2024. "“Whatever It Takes!” How tonality of TV-news affected government bond yield spreads during the European debt crisis," European Journal of Political Economy, Elsevier, vol. 82(C).
    9. Robert Hogenraad, 2021. "The way of visionaries: foresight and imagination, computed," Quality & Quantity: International Journal of Methodology, Springer, vol. 55(5), pages 1631-1660, October.
    10. Katarina Böttcher & Kerstin Lopatta, 2020. "Gender-Sensitive Language in German Annual Reports," Journal of Management and Sustainability, Canadian Center of Science and Education, vol. 8(4), pages 1-1, March.
    11. Rauh, Christian, 2018. "Validating a sentiment dictionary for German political language—a workbench note," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 15(4), pages 319-343.
    12. Miklós Sebők & Orsolya Ring & Márk György Kis & Martin Balázs Bánóczy & Ágnes Dinnyés, 2024. "The geopolitics of vaccine media representation in Orbán’s Hungary—an AI-supported sentiment analysis," Journal of Computational Social Science, Springer, vol. 7(3), pages 2897-2920, December.
    13. Zobel, Malisa & Lehmann, Pola, 2018. "Positions and saliency of immigration in party manifestos: A novel dataset using crowd coding," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 57(4), pages 1056-1083.
    14. Dimitrios Kydros & Maria Argyropoulou & Vasiliki Vrana, 2021. "A Content and Sentiment Analysis of Greek Tweets during the Pandemic," Sustainability, MDPI, vol. 13(11), pages 1-21, May.
    15. Elif Günalan & Saadet Turhan & Betül Yıldırım Çavak & İrem Kaya Cebioğlu & Özge Çonak, 2022. "The Evaluation of Videos about Branched-Chain Amino Acids Supplements on YouTube ™ : A Multi-Approach Study," IJERPH, MDPI, vol. 19(24), pages 1-15, December.
    16. Hirsch, Patrick & Köhler, Ekkehard A. & Feld, Lars P. & Thomas, Tobias, 2020. ""Whatever it takes!": How tonality of TV-news affects government bond yield spreads during crises," Freiburg Discussion Papers on Constitutional Economics 20/9, Walter Eucken Institut e.V..
    17. Shrub, Yuliya & Rieger, Jonas & Müller, Henrik & Jentsch, Carsten, 2022. "Text data rule - don't they? A study on the (additional) information of Handelsblatt data for nowcasting German GDP in comparison to established economic indicators," Ruhr Economic Papers 964, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    18. Robert Hogenraad, 2019. "Fear in the West: a sentiment analysis using a computer-readable “Fear Index”," Quality & Quantity: International Journal of Methodology, Springer, vol. 53(3), pages 1239-1261, May.
    19. Hiroki Takikawa & Takuto Sakamoto, 2020. "The moral–emotional foundations of political discourse: a comparative analysis of the speech records of the U.S. and the Japanese legislatures," Quality & Quantity: International Journal of Methodology, Springer, vol. 54(2), pages 547-566, April.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gavin Abercrombie & Riza Batista-Navarro, 2020. "Sentiment and position-taking analysis of parliamentary debates: a systematic literature review," Journal of Computational Social Science, Springer, vol. 3(1), pages 245-270, April.
    2. Ping-Yu Hsu & Hong-Tsuen Lei & Shih-Hsiang Huang & Teng Hao Liao & Yao-Chung Lo & Chin-Chun Lo, 2019. "Effects of sentiment on recommendations in social network," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(2), pages 253-262, June.
    3. Diaf, Sami & Döpke, Jörg & Fritsche, Ulrich & Rockenbach, Ida, 2022. "Sharks and minnows in a shoal of words: Measuring latent ideological positions based on text mining techniques," European Journal of Political Economy, Elsevier, vol. 75(C).
    4. Greene, Zac & Ceron, Andrea & Schumacher, Gijs & Fazekas, Zoltan, 2016. "The Nuts and Bolts of Automated Text Analysis. Comparing Different Document Pre-Processing Techniques in Four Countries," OSF Preprints ghxj8, Center for Open Science.
    5. Simon Albrecht & Bernhard Lutz & Dirk Neumann, 2020. "The behavior of blockchain ventures on Twitter as a determinant for funding success," Electronic Markets, Springer;IIM University of St. Gallen, vol. 30(2), pages 241-257, June.
    6. H. Andrew Schwartz & Lyle H. Ungar, 2015. "Data-Driven Content Analysis of Social Media," The ANNALS of the American Academy of Political and Social Science, , vol. 659(1), pages 78-94, May.
    7. H Andrew Schwartz & Johannes C Eichstaedt & Margaret L Kern & Lukasz Dziurzynski & Stephanie M Ramones & Megha Agrawal & Achal Shah & Michal Kosinski & David Stillwell & Martin E P Seligman & Lyle H U, 2013. "Personality, Gender, and Age in the Language of Social Media: The Open-Vocabulary Approach," PLOS ONE, Public Library of Science, vol. 8(9), pages 1-16, September.
    8. Alexander Herzog & Slava Mikhaylov, 2010. "A new Database of Parliamentary Debates in Ireland, 1922--2008," The Institute for International Integration Studies Discussion Paper Series iiisdp338, IIIS, revised Jul 2010.
    9. Rodrigo Zamith & Seth C. Lewis, 2015. "Content Analysis and the Algorithmic Coder," The ANNALS of the American Academy of Political and Social Science, , vol. 659(1), pages 307-318, May.
    10. Kostovicova Denisa & Kerr Rachel & Sokolić Ivor & Fairey Tiffany & Redwood Henry & Subotić Jelena, 2022. "The “Digital Turn” in Transitional Justice Research: Evaluating Image and Text as Data in the Western Balkans," Comparative Southeast European Studies, De Gruyter, vol. 70(1), pages 24-46, March.
    11. Anna Calissano & Simone Vantini & Marika Arena, 2020. "Monitoring rare categories in sentiment and opinion analysis: a Milan mega event on Twitter platform," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 29(4), pages 787-812, December.
    12. Sami Diaf & Jörg Döpke & Ulrich Fritsche & Ida Rockenbach, 2020. "Sharks and minnows in a shoal of words: Measuring latent ideological positions of German economic research institutes based on text mining techniques," Macroeconomics and Finance Series 202001, University of Hamburg, Department of Socioeconomics.
    13. Weiss, Max & Zoorob, Michael, 2021. "Political frames of public health crises: Discussing the opioid epidemic in the US Congress," Social Science & Medicine, Elsevier, vol. 281(C).
    14. Pierre-Marc Daigneault & Dominic Duval & Louis M. Imbeau, 2018. "Supervised scaling of semi-structured interview transcripts to characterize the ideology of a social policy reform," Quality & Quantity: International Journal of Methodology, Springer, vol. 52(5), pages 2151-2162, September.
    15. David M. Goldberg & Nohel Zaman & Arin Brahma & Mariano Aloiso, 2022. "Are mortgage loan closing delay risks predictable? A predictive analysis using text mining on discussion threads," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(3), pages 419-437, March.
    16. Gabriele Ranco & Ilaria Bordino & Giacomo Bormetti & Guido Caldarelli & Fabrizio Lillo & Michele Treccani, 2014. "Coupling news sentiment with web browsing data improves prediction of intra-day price dynamics," Papers 1412.3948, arXiv.org, revised Dec 2015.
    17. Adriana Bunea & Raimondas Ibenskas, 2015. "Quantitative text analysis and the study of EU lobbying and interest groups," European Union Politics, , vol. 16(3), pages 429-455, September.
    18. Rebecca Cordell & Kristian Skrede Gleditsch & Florian G Kern & Laura Saavedra-Lux, 2020. "Measuring institutional variation across American Indian constitutions using automated content analysis," Journal of Peace Research, Peace Research Institute Oslo, vol. 57(6), pages 777-788, November.
    19. Born, Andreas & Janssen, Aljoscha, 2020. "Does a District-Vote Matter for the Behavior of Politicians? A Textual Analysis of Parliamentary Speeches," Working Paper Series 1320, Research Institute of Industrial Economics.
    20. Weifeng Zhong, 2016. "The candidates in their own words: A textual analysis of 2016 president primary debates," AEI Economic Perspectives, American Enterprise Institute, April.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:qualqt:v:51:y:2017:i:6:d:10.1007_s11135-016-0412-4. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.