IDEAS home Printed from https://ideas.repec.org/a/spr/nathaz/v113y2022i1d10.1007_s11069-022-05307-w.html
   My bibliography  Save this article

Accuracy of a pre-trained sentiment analysis (SA) classification model on tweets related to emergency response and early recovery assessment: the case of 2019 Albanian earthquake

Author

Listed:
  • Diana Contreras

    (Cardiff University
    Newcastle University)

  • Sean Wilkinson

    (Newcastle University)

  • Evangeline Alterman

    (Auckland University)

  • Javier Hervás

Abstract

Traditionally, earthquake impact assessments have been made via fieldwork by non-governmental organisations (NGO's) sponsored data collection; however, this approach is time-consuming, expensive and often limited. Recently, social media (SM) has become a valuable tool for quickly collecting large amounts of first-hand data after a disaster and shows great potential for decision-making. Nevertheless, extracting meaningful information from SM is an ongoing area of research. This paper tests the accuracy of the pre-trained sentiment analysis (SA) model developed by the no-code machine learning platform MonkeyLearn using the text data related to the emergency response and early recovery phase of the three major earthquakes that struck Albania on the 26th November 2019. These events caused 51 deaths, 3000 injuries and extensive damage. We obtained 695 tweets with the hashtags: #Albania #AlbanianEarthquake, and #albanianearthquake from the 26th November 2019 to the 3rd February 2020. We used these data to test the accuracy of the pre-trained SA classification model developed by MonkeyLearn to identify polarity in text data. This test explores the feasibility to automate the classification process to extract meaningful information from text data from SM in real-time in the future. We tested the no-code machine learning platform's performance using a confusion matrix. We obtained an overall accuracy (ACC) of 63% and a misclassification rate of 37%. We conclude that the ACC of the unsupervised classification is sufficient for a preliminary assessment, but further research is needed to determine if the accuracy is improved by customising the training model of the machine learning platform.

Suggested Citation

  • Diana Contreras & Sean Wilkinson & Evangeline Alterman & Javier Hervás, 2022. "Accuracy of a pre-trained sentiment analysis (SA) classification model on tweets related to emergency response and early recovery assessment: the case of 2019 Albanian earthquake," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 113(1), pages 403-421, August.
    • Handle: RePEc:spr:nathaz:v:113:y:2022:i:1:d:10.1007_s11069-022-05307-w
    DOI: 10.1007/s11069-022-05307-w
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11069-022-05307-w
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11069-022-05307-w?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Simon, Tomer & Goldberg, Avishay & Adini, Bruria, 2015. "Socializing in emergencies—A review of the use of social media in emergency situations," International Journal of Information Management, Elsevier, vol. 35(5), pages 609-619.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Luis-Millán González & José Devís-Devís & Maite Pellicer-Chenoll & Miquel Pans & Alberto Pardo-Ibañez & Xavier García-Massó & Fernanda Peset & Fernanda Garzón-Farinós & Víctor Pérez-Samaniego, 2021. "The Impact of COVID-19 on Sport in Twitter: A Quantitative and Qualitative Content Analysis," IJERPH, MDPI, vol. 18(9), pages 1-20, April.
    2. Ni, Zi-jian & Rong, Lili & Wang, Ning & Cao, Shuo, 2019. "Knowledge model for emergency response based on contingency planning system of China," International Journal of Information Management, Elsevier, vol. 46(C), pages 10-22.
    3. Moran Bodas & Morel Ragoler & Yossi Rabby & Esther Krasner, 2021. "The Effect of Risk Communication on Public Behavior to Non-Conventional Terrorism—Randomized Control Trial," IJERPH, MDPI, vol. 19(1), pages 1-20, December.
    4. Li, Lifang & Zhang, Qingpeng & Tian, Jun & Wang, Haolin, 2018. "Characterizing information propagation patterns in emergencies: A case study with Yiliang Earthquake," International Journal of Information Management, Elsevier, vol. 38(1), pages 34-41.
    5. Elbanna, Amany & Bunker, Deborah & Levine, Linda & Sleigh, Anthony, 2019. "Emergency management in the changing world of social media: Framing the research agenda with the stakeholders through engaged scholarship," International Journal of Information Management, Elsevier, vol. 47(C), pages 112-120.
    6. Al-Omoush, Khaled Saleh & Garrido, Rubén & Cañero, Julio, 2023. "The impact of government use of social media and social media contradictions on trust in government and citizens’ attitudes in times of crisis," Journal of Business Research, Elsevier, vol. 159(C).
    7. Dionne Mitcham & Morgan Taylor & Curtis Harris, 2021. "Utilizing Social Media for Information Dispersal during Local Disasters: The Communication Hub Framework for Local Emergency Management," IJERPH, MDPI, vol. 18(20), pages 1-16, October.
    8. Umar Ali Bukar & Fatimah Sidi & Marzanah A. Jabar & Rozi Nor Haizan Nor & Salfarina Abdullah & Iskandar Ishak & Mustafa Alabadla & Ali Alkhalifah, 2022. "How Advanced Technological Approaches Are Reshaping Sustainable Social Media Crisis Management and Communication: A Systematic Review," Sustainability, MDPI, vol. 14(10), pages 1-26, May.
    9. Boyang Shi & Weixiang Huang & Yuanyuan Dang & Wenhui Zhou, 2024. "Leveraging social media data for pandemic detection and prediction," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-18, December.
    10. Feng, Xinhang & Jiang, Yanli & Gai, Wenmei, 2024. "Rural community response to accidental toxic gas release: An individual emergency response model during self-organized evacuations," Reliability Engineering and System Safety, Elsevier, vol. 248(C).
    11. Dominik Emanuel Froehlich, 2021. "Career Networks in Shock: An Agenda for in-COVID/Post-COVID Career-Related Social Capital," Merits, MDPI, vol. 1(1), pages 1-10, November.
    12. Kerstin K. Zander & Jonas Rieskamp & Milad Mirbabaie & Mamoun Alazab & Duy Nguyen, 2023. "Responses to heat waves: what can Twitter data tell us?," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 116(3), pages 3547-3564, April.
    13. Pelen, Neslihan Nesliye & Gölgeli, Meltem, 2022. "Vector-borne disinformation during disasters and emergencies," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 596(C).
    14. Son, Jaebong & Lee, Hyung Koo & Jin, Sung & Lee, Jintae, 2019. "Content features of tweets for effective communication during disasters: A media synchronicity theory perspective," International Journal of Information Management, Elsevier, vol. 45(C), pages 56-68.
    15. Jamali, Mehdi & Nejat, Ali & Ghosh, Souparno & Jin, Fang & Cao, Guofeng, 2019. "Social media data and post-disaster recovery," International Journal of Information Management, Elsevier, vol. 44(C), pages 25-37.
    16. Agarwal, Shweta & Kumar, Shailendra & Goel, Utkarsh, 2019. "Stock market response to information diffusion through internet sources: A literature review," International Journal of Information Management, Elsevier, vol. 45(C), pages 118-131.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:nathaz:v:113:y:2022:i:1:d:10.1007_s11069-022-05307-w. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.