IDEAS home Printed from https://ideas.repec.org/a/gam/jmathe/v10y2022i24p4664-d998091.html
   My bibliography  Save this article

A Comprehensive Analysis of Transformer-Deep Neural Network Models in Twitter Disaster Detection

Author

Listed:
  • Vimala Balakrishnan

    (Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur 50603, Malaysia)

  • Zhongliang Shi

    (Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur 50603, Malaysia)

  • Chuan Liang Law

    (Malayan Banking Berhad, Kuala Lumpur 50050, Malaysia)

  • Regine Lim

    (Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur 50603, Malaysia)

  • Lee Leng Teh

    (Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur 50603, Malaysia)

  • Yue Fan

    (Faculty of Computer Science and Information Technology, Universiti Malaya, Kuala Lumpur 50603, Malaysia)

  • Jeyarani Periasamy

    (Faculty of Data Science and Information Technology, INTI International University, Nilai 71800, Malaysia)

Abstract

Social media platforms such as Twitter are a vital source of information during major events, such as natural disasters. Studies attempting to automatically detect textual communications have mostly focused on machine learning and deep learning algorithms. Recent evidence shows improvement in disaster detection models with the use of contextual word embedding techniques (i.e., transformers) that take the context of a word into consideration, unlike the traditional context-free techniques; however, studies regarding this model are scant. To this end, this paper investigates a selection of ensemble learning models by merging transformers with deep neural network algorithms to assess their performance in detecting informative and non-informative disaster-related Twitter communications. A total of 7613 tweets were used to train and test the models. Results indicate that the ensemble models consistently yield good performance results, with F-score values ranging between 76% and 80%. Simpler transformer variants, such as ELECTRA and Talking-Heads Attention, yielded comparable and superior results compared to the computationally expensive BERT, with F-scores ranging from 80% to 84%, especially when merged with Bi-LSTM. Our findings show that the newer and simpler transformers can be used effectively, with less computational costs, in detecting disaster-related Twitter communications.

Suggested Citation

  • Vimala Balakrishnan & Zhongliang Shi & Chuan Liang Law & Regine Lim & Lee Leng Teh & Yue Fan & Jeyarani Periasamy, 2022. "A Comprehensive Analysis of Transformer-Deep Neural Network Models in Twitter Disaster Detection," Mathematics, MDPI, vol. 10(24), pages 1-14, December.
  • Handle: RePEc:gam:jmathe:v:10:y:2022:i:24:p:4664-:d:998091
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7390/10/24/4664/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/2227-7390/10/24/4664/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Guizhe Song & Degen Huang, 2021. "A Sentiment-Aware Contextual Model for Real-Time Disaster Prediction Using Twitter Data," Future Internet, MDPI, vol. 13(7), pages 1-15, June.
    2. Ragini, J. Rexiline & Anand, P.M. Rubesh & Bhaskar, Vidhyacharan, 2018. "Big data analytics for disaster response and recovery through sentiment analysis," International Journal of Information Management, Elsevier, vol. 42(C), pages 13-24.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Gabriel Bercaru & Ciprian-Octavian Truică & Costin-Gabriel Chiru & Traian Rebedea, 2023. "Improving Intent Classification Using Unlabeled Data from Large Corpora," Mathematics, MDPI, vol. 11(3), pages 1-20, February.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. de Camargo Fiorini, Paula & Roman Pais Seles, Bruno Michel & Chiappetta Jabbour, Charbel Jose & Barberio Mariano, Enzo & de Sousa Jabbour, Ana Beatriz Lopes, 2018. "Management theory and big data literature: From a review to a research agenda," International Journal of Information Management, Elsevier, vol. 43(C), pages 112-129.
    2. Carlos Carrasco-Farré, 2022. "The fingerprints of misinformation: how deceptive content differs from reliable sources in terms of cognitive effort and appeal to emotions," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-18, December.
    3. Acharya, Abhilash & Singh, Sanjay Kumar & Pereira, Vijay & Singh, Poonam, 2018. "Big data, knowledge co-creation and decision making in fashion industry," International Journal of Information Management, Elsevier, vol. 42(C), pages 90-101.
    4. Umar Ali Bukar & Fatimah Sidi & Marzanah A. Jabar & Rozi Nor Haizan Nor & Salfarina Abdullah & Iskandar Ishak & Mustafa Alabadla & Ali Alkhalifah, 2022. "How Advanced Technological Approaches Are Reshaping Sustainable Social Media Crisis Management and Communication: A Systematic Review," Sustainability, MDPI, vol. 14(10), pages 1-26, May.
    5. Muhammad Ashraf Fauzi, 2023. "Social media in disaster management: review of the literature and future trends through bibliometric analysis," Natural Hazards: Journal of the International Society for the Prevention and Mitigation of Natural Hazards, Springer;International Society for the Prevention and Mitigation of Natural Hazards, vol. 118(2), pages 953-975, September.
    6. Ni, Zi-jian & Rong, Lili & Wang, Ning & Cao, Shuo, 2019. "Knowledge model for emergency response based on contingency planning system of China," International Journal of Information Management, Elsevier, vol. 46(C), pages 10-22.
    7. Abhinav Kumar & Jyoti Prakash Singh & Yogesh K. Dwivedi & Nripendra P. Rana, 2022. "A deep multi-modal neural network for informative Twitter content classification during emergencies," Annals of Operations Research, Springer, vol. 319(1), pages 791-822, December.
    8. Turgut Acikara & Bo Xia & Tan Yigitcanlar & Carol Hon, 2023. "Contribution of Social Media Analytics to Disaster Response Effectiveness: A Systematic Review of the Literature," Sustainability, MDPI, vol. 15(11), pages 1-50, May.
    9. Harri Raisio & Alisa Puustinen & Juha Lindell, 2022. "#StrongTogether? Qualitative Sentiment Analysis of Social Media Reactions to Disaster Volunteering during a Forest Fire in Finland," Sustainability, MDPI, vol. 14(7), pages 1-18, March.
    10. Gupta, Shivam & Kar, Arpan Kumar & Baabdullah, Abdullah & Al-Khowaiter, Wassan A.A., 2018. "Big data with cognitive computing: A review for the future," International Journal of Information Management, Elsevier, vol. 42(C), pages 78-89.
    11. Paras Bhatt & Naga Vemprala & Rohit Valecha & Govind Hariharan & H. Raghav Rao, 2023. "User Privacy, Surveillance and Public Health during COVID-19 – An Examination of Twitterverse," Information Systems Frontiers, Springer, vol. 25(5), pages 1667-1682, October.
    12. Siqing Shan & Xijie Ju & Yigang Wei & Xin Wen, 2022. "Concerned or Apathetic? Using Social Media Platform (Twitter) to Gauge the Public Awareness about Wildlife Conservation: A Case Study of the Illegal Rhino Trade," IJERPH, MDPI, vol. 19(11), pages 1-21, June.
    13. Cheng-Chun Lee & Mikel Maron & Ali Mostafavi, 2022. "Community-scale big data reveals disparate impacts of the Texas winter storm of 2021 and its managed power outage," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-12, December.
    14. Gozuacik, Necip & Sakar, C. Okan & Ozcan, Sercan, 2023. "Technological forecasting based on estimation of word embedding matrix using LSTM networks," Technological Forecasting and Social Change, Elsevier, vol. 191(C).
    15. Sachin Modgil & Rohit Kumar Singh & Cyril Foropon, 2022. "Quality management in humanitarian operations and disaster relief management: a review and future research directions," Annals of Operations Research, Springer, vol. 319(1), pages 1045-1098, December.
    16. Elbanna, Amany & Bunker, Deborah & Levine, Linda & Sleigh, Anthony, 2019. "Emergency management in the changing world of social media: Framing the research agenda with the stakeholders through engaged scholarship," International Journal of Information Management, Elsevier, vol. 47(C), pages 112-120.
    17. M. R. Mahendrini Fernando Ariyachandra & Gayan Wedawatta, 2023. "Digital Twin Smart Cities for Disaster Risk Management: A Review of Evolving Concepts," Sustainability, MDPI, vol. 15(15), pages 1-25, August.
    18. Seddigh, Mohammad Reza & Targholizadeh, Aida & Shokouhyar, Sajjad & Shokoohyar, Sina, 2023. "Social media and expert analysis cast light on the mechanisms of underlying problems in pharmaceutical supply chain: An exploratory approach," Technological Forecasting and Social Change, Elsevier, vol. 191(C).
    19. Martínez-Rojas, María & Pardo-Ferreira, María del Carmen & Rubio-Romero, Juan Carlos, 2018. "Twitter as a tool for the management and analysis of emergency situations: A systematic literature review," International Journal of Information Management, Elsevier, vol. 43(C), pages 196-208.
    20. Carlos Galera-Zarco & Goulielmos Floros, 2024. "A deep learning approach to improve built asset operations and disaster management in critical events: an integrative simulation model for quicker decision making," Annals of Operations Research, Springer, vol. 339(1), pages 573-612, August.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jmathe:v:10:y:2022:i:24:p:4664-:d:998091. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.