IDEAS home Printed from https://ideas.repec.org/a/sae/anname/v659y2015i1p95-107.html
   My bibliography  Save this article

Signals of Public Opinion in Online Communication

Author

Listed:
  • Sandra González-Bailón
  • Georgios Paltoglou

Abstract

This study offers a systematic comparison of automated content analysis tools. The ability of different lexicons to correctly identify affective tone (e.g., positive vs. negative) is assessed in different social media environments. Our comparisons examine the reliability and validity of publicly available, off-the-shelf classifiers. We use datasets from a range of online sources that vary in the diversity and formality of the language used, and we apply different classifiers to extract information about the affective tone in these datasets. We first measure agreement (reliability test) and then compare their classifications with the benchmark of human coding (validity test). Our analyses show that validity and reliability vary with the formality and diversity of the text; we also show that ready-to-use methods leave much space for improvement when analyzing domain-specific content and that a machine-learning approach offers more accurate predictions across communication domains.

Suggested Citation

  • Sandra González-Bailón & Georgios Paltoglou, 2015. "Signals of Public Opinion in Online Communication," The ANNALS of the American Academy of Political and Social Science, , vol. 659(1), pages 95-107, May.
    • Handle: RePEc:sae:anname:v:659:y:2015:i:1:p:95-107
    DOI: 10.1177/0002716215569192
    as

    Download full text from publisher

    File URL: https://journals.sagepub.com/doi/10.1177/0002716215569192
    Download Restriction: no
    File URL: https://libkey.io/10.1177/0002716215569192?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Mike Thelwall & Kevan Buckley & Georgios Paltoglou & Di Cai & Arvid Kappas, 2010. "Sentiment strength detection in short informal text," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 61(12), pages 2544-2558, December.
    2. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2011. "Sentiment in Twitter events," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 62(2), pages 406-418, February.
    3. Mike Thelwall & Pardeep Sud & Farida Vis, 2012. "Commenting on YouTube videos: From guatemalan rock to El Big Bang," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 63(3), pages 616-629, March.
    4. Mike Thelwall & Kevan Buckley & Georgios Paltoglou, 2011. "Sentiment in Twitter events," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 62(2), pages 406-418, February.
    5. Mike Thelwall & Kevan Buckley & Georgios Paltoglou & Di Cai & Arvid Kappas, 2010. "Sentiment strength detection in short informal text," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 61(12), pages 2544-2558, December.
    6. Mike Thelwall & Pardeep Sud & Farida Vis, 2012. "Commenting on YouTube videos: From guatemalan rock to El Big Bang," Journal of the American Society for Information Science and Technology, Association for Information Science & Technology, vol. 63(3), pages 616-629, March.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Andrew Todd & James Bowden & Yashar Moshfeghi, 2024. "Text‐based sentiment analysis in finance: Synthesising the existing literature and exploring future directions," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 31(1), March.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Ma, Jie & Tse, Ying Kei & Wang, Xiaojun & Zhang, Minhao, 2019. "Examining customer perception and behaviour through social media research – An empirical study of the United Airlines overbooking crisis," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 127(C), pages 192-205.
    2. Ping-Yu Hsu & Hong-Tsuen Lei & Shih-Hsiang Huang & Teng Hao Liao & Yao-Chung Lo & Chin-Chun Lo, 2019. "Effects of sentiment on recommendations in social network," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(2), pages 253-262, June.
    3. Heleen Brans & Bert Scholtens, 2020. "Under his thumb the effect of president Donald Trump’s Twitter messages on the US stock market," PLOS ONE, Public Library of Science, vol. 15(3), pages 1-11, March.
    4. Oleg S. Nagornyy & Olessia Y. Koltsova, 2017. "Mining Media Topics Perceived as Social Problems by Online Audiences: Use of a Data Mining Approach in Sociology," HSE Working papers WP BRP 74/SOC/2017, National Research University Higher School of Economics.
    5. Christopher Courtney & Supradeep Dutta & Yong Li, 2017. "Resolving Information Asymmetry: Signaling, Endorsement, and Crowdfunding Success," Entrepreneurship Theory and Practice, , vol. 41(2), pages 265-290, March.
    6. Sanmitra Bhattacharya & Padmini Srinivasan & Phil Polgreen, 2014. "Engagement with Health Agencies on Twitter," PLOS ONE, Public Library of Science, vol. 9(11), pages 1-12, November.
    7. Lipizzi, Carlo & Iandoli, Luca & Ramirez Marquez, José Emmanuel, 2015. "Extracting and evaluating conversational patterns in social media: A socio-semantic analysis of customers’ reactions to the launch of new products using Twitter streams," International Journal of Information Management, Elsevier, vol. 35(4), pages 490-503.
    8. Hajar Sotudeh & Zeinab Saber & Farzin Ghanbari Aloni & Mahdieh Mirzabeigi & Farshad Khunjush, 2022. "A longitudinal study of the evolution of opinions about open access and its main features: a twitter sentiment analysis," Scientometrics, Springer;Akadémiai Kiadó, vol. 127(10), pages 5587-5611, October.
    9. Luis-Millán González & José Devís-Devís & Maite Pellicer-Chenoll & Miquel Pans & Alberto Pardo-Ibañez & Xavier García-Massó & Fernanda Peset & Fernanda Garzón-Farinós & Víctor Pérez-Samaniego, 2021. "The Impact of COVID-19 on Sport in Twitter: A Quantitative and Qualitative Content Analysis," IJERPH, MDPI, vol. 18(9), pages 1-20, April.
    10. Karin Sim Smith & Richard McCreadie & Craig Macdonald & Iadh Ounis, 2018. "Regional Sentiment Bias in Social Media Reporting During Crises," Information Systems Frontiers, Springer, vol. 20(5), pages 1013-1025, October.
    11. Beatriz Barros & Ana Fernández-Zubieta & Raul Fidalgo-Merino & Francisco Triguero, 2018. "Scientific knowledge percolation process and social impact: A case study on the biotechnology and microbiology perceptions on Twitter," Science and Public Policy, Oxford University Press, vol. 45(6), pages 804-814.
    12. Müller-Hansen, Finn & Lee, Yuan Ting & Callaghan, Max & Jankin, Slava & Minx, Jan C., 2022. "The German coal debate on Twitter: Reactions to a corporate policy process," Energy Policy, Elsevier, vol. 169(C).
    13. Mike Thelwall & David Foster, 2021. "Male or female gender‐polarized YouTube videos are less viewed," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 72(12), pages 1545-1557, December.
    14. Daesik Kim & Chung Joo Chung & Kihong Eom, 2022. "Measuring Online Public Opinion for Decision Making: Application of Deep Learning on Political Context," Sustainability, MDPI, vol. 14(7), pages 1-16, March.
    15. Gabriele Ranco & Ilaria Bordino & Giacomo Bormetti & Guido Caldarelli & Fabrizio Lillo & Michele Treccani, 2014. "Coupling news sentiment with web browsing data improves prediction of intra-day price dynamics," Papers 1412.3948, arXiv.org, revised Dec 2015.
    16. Tadić, Bosiljka & Mitrović Dankulov, Marija & Melnik, Roderick, 2023. "Evolving cycles and self-organised criticality in social dynamics," Chaos, Solitons & Fractals, Elsevier, vol. 171(C).
    17. Thomas T. Hills & Eugenio Proto & Daniel Sgroi & Chanuki Illushka Seresinhe, 2019. "Historical analysis of national subjective wellbeing using millions of digitized books," Nature Human Behaviour, Nature, vol. 3(12), pages 1271-1275, December.
    18. Cohen, Scott & Stienmetz, Jason & Hanna, Paul & Humbracht, Michael & Hopkins, Debbie, 2020. "Shadowcasting tourism knowledge through media: Self-driving sex cars?," Annals of Tourism Research, Elsevier, vol. 85(C).
    19. Zhang, Xuetong & Zhang, Weiguo, 2023. "Information asymmetry, sentiment interactions, and asset price," The North American Journal of Economics and Finance, Elsevier, vol. 67(C).
    20. Indy Wijngaards & Martijn Burger & Job van Exel, 2019. "The promise of open survey questions—The validation of text-based job satisfaction measures," PLOS ONE, Public Library of Science, vol. 14(12), pages 1-22, December.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:sae:anname:v:659:y:2015:i:1:p:95-107. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: SAGE Publications (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.