IDEAS home Printed from https://ideas.repec.org/a/spr/jcsosc/v7y2024i3d10.1007_s42001-024-00326-y.html
   My bibliography  Save this article

A comprehensive review of visual–textual sentiment analysis from social media networks

Author

Listed:
  • Israa Khalaf Salman Al-Tameemi

    (University of Tabriz)

  • Mohammad-Reza Feizi-Derakhshi

    (University of Tabriz)

  • Saeed Pashazadeh

    (University of Tabriz)

  • Mohammad Asadpour

    (University of Tabriz)

Abstract

Social media networks have become a significant aspect of people’s lives, serving as a platform for their ideas, opinions and emotions. Consequently, automated sentiment analysis (SA) is critical for recognising people’s feelings in ways other information sources cannot. The analysis of these feelings revealed various applications, including brand evaluations, YouTube film reviews and healthcare applications. As social media continues to develop, people publish vast quantities of information in various formats, like text, pictures, audio, and video. Thus, traditional SA algorithms have become limited, as they do not consider the expressiveness of other modalities. By including such characteristics from various material sources, these multimodal data streams provide new opportunities for optimising the expected results beyond text-based SA. Our study focuses on the forefront field of multimodal SA, which examines visual and textual data posted on social media networks. Many people are more likely to utilise this information to express themselves on these platforms. To serve as a resource for academics in this rapidly growing field, we introduce a comprehensive overview of textual and visual SA, including data pre-processing, feature extraction techniques, sentiment benchmark datasets, and the efficacy of multiple classification methodologies suited to each field. We also provide a brief introduction of the most frequently utilised data fusion strategies and a summary of existing research on visual–textual SA. Finally, we highlight the most significant challenges and investigate several important sentiment applications.

Suggested Citation

  • Israa Khalaf Salman Al-Tameemi & Mohammad-Reza Feizi-Derakhshi & Saeed Pashazadeh & Mohammad Asadpour, 2024. "A comprehensive review of visual–textual sentiment analysis from social media networks," Journal of Computational Social Science, Springer, vol. 7(3), pages 2767-2838, December.
    • Handle: RePEc:spr:jcsosc:v:7:y:2024:i:3:d:10.1007_s42001-024-00326-y
    DOI: 10.1007/s42001-024-00326-y
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s42001-024-00326-y
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s42001-024-00326-y?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mohammad Saber Iraji & Mohammad-Reza Feizi-Derakhshi & Jafar Tanha & Adil Mehmood Khan, 2021. "COVID-19 Detection Using Deep Convolutional Neural Networks and Binary Differential Algorithm-Based Feature Selection from X-Ray Images," Complexity, Hindawi, vol. 2021, pages 1-10, October.
    2. Nikzad-Khasmakhi, N. & Balafar, M.A. & Reza Feizi-Derakhshi, M. & Motamed, Cina, 2021. "BERTERS: Multimodal representation learning for expert recommendation system with transformers and graph embeddings," Chaos, Solitons & Fractals, Elsevier, vol. 151(C).
    3. Hedayati , Amin & Hedayati , Moein & Esfandyari, Morteza, 2016. "Stock market index prediction using artificial neural network," Journal of Economics, Finance and Administrative Science, Universidad ESAN, vol. 21(41), pages 89-93.
    4. Israa K. Salman Al-Tameemi & Mohammad-Reza Feizi-Derakhshi & Saeed Pashazadeh & Mohammad Asadpour & Roberto Natella, 2023. "An Efficient Sentiment Classification Method with the Help of Neighbors and a Hybrid of RNN Models," Complexity, Hindawi, vol. 2023, pages 1-14, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Caporale, Guglielmo Maria & Gil-Alana, Luis Alberiko & Poza, Carlos, 2022. "The COVID-19 pandemic and the degree of persistence of US stock prices and bond yields," The Quarterly Review of Economics and Finance, Elsevier, vol. 86(C), pages 118-123.
    2. Priyank Sonkiya & Vikas Bajpai & Anukriti Bansal, 2021. "Stock price prediction using BERT and GAN," Papers 2107.09055, arXiv.org.
    3. Ehsan Hoseinzade & Saman Haratizadeh & Arash Khoeini, 2019. "U-CNNpred: A Universal CNN-based Predictor for Stock Markets," Papers 1911.12540, arXiv.org.
    4. Semenoglou, Artemios-Anargyros & Spiliotis, Evangelos & Makridakis, Spyros & Assimakopoulos, Vassilios, 2021. "Investigating the accuracy of cross-learning time series forecasting methods," International Journal of Forecasting, Elsevier, vol. 37(3), pages 1072-1084.
    5. Sangyeon Kim & Myungjoo Kang, 2019. "Financial series prediction using Attention LSTM," Papers 1902.10877, arXiv.org.
    6. Manel Hamdi & Walid Chkili, 2019. "An artificial neural network augmented GARCH model for Islamic stock market volatility: Do asymmetry and long memory matter?," Working Papers 13, Economic Research Forum, revised 21 Aug 2019.
    7. Po Yun & Chen Zhang & Yaqi Wu & Xianzi Yang & Zulfiqar Ali Wagan, 2020. "A Novel Extended Higher-Order Moment Multi-Factor Framework for Forecasting the Carbon Price: Testing on the Multilayer Long Short-Term Memory Network," Sustainability, MDPI, vol. 12(5), pages 1-16, March.
    8. Rayadurgam, Vikram Chandramouli & Mangalagiri, Jayasree, 2023. "Does inclusion of GARCH variance in deep learning models improve financial contagion prediction?," Finance Research Letters, Elsevier, vol. 54(C).
    9. Miguel A. Jaramillo-Morán & Agustín García-García, 2019. "Applying Artificial Neural Networks to Forecast European Union Allowance Prices: The Effect of Information from Pollutant-Related Sectors," Energies, MDPI, vol. 12(23), pages 1-18, November.
    10. Federico Mecchia & Marcellino Gaudenzi, 2022. "The dynamics of the prices of the companies of the STOXX Europe 600 Index through the logit model and neural network," Papers 2206.09899, arXiv.org.
    11. Hongbing OUYANG & Xiaolu WEI & Qiufeng WU, 2020. "Stock Index Pattern Discovery via Toeplitz Inverse Covariance-based Clustering," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(2), pages 58-72, July.
    12. Xiaoping Du & Lelai Deng, 2018. "Current Core Competencies Trend of Small to Medium Enterprises (SMEs) in China—A Concurrent Comprehensive Evaluation and Active Learning Study of Newly Listed Chinese Stocks from 2015 through 2017," Data, MDPI, vol. 3(3), pages 1-22, July.
    13. Ishu Gupta & Tarun Kumar Madan & Sukhman Singh & Ashutosh Kumar Singh, 2022. "HiSA-SMFM: Historical and Sentiment Analysis based Stock Market Forecasting Model," Papers 2203.08143, arXiv.org.
    14. Heni Boubaker & Giorgio Canarella & Rangan Gupta & Stephen M. Miller, 2023. "A Hybrid ARFIMA Wavelet Artificial Neural Network Model for DJIA Index Forecasting," Computational Economics, Springer;Society for Computational Economics, vol. 62(4), pages 1801-1843, December.
    15. Lin, Yu & Yan, Yan & Xu, Jiali & Liao, Ying & Ma, Feng, 2021. "Forecasting stock index price using the CEEMDAN-LSTM model," The North American Journal of Economics and Finance, Elsevier, vol. 57(C).
    16. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    17. Zhenni Jin & Kun Guo & Yi Sun & Lin Lai & Zhewen Liao, 2020. "The industrial asymmetry of the stock price prediction with investor sentiment: Based on the comparison of predictive effects with SVR," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(7), pages 1166-1178, November.
    18. Jessie Sun, 2019. "A Stock Selection Method Based on Earning Yield Forecast Using Sequence Prediction Models," Papers 1905.04842, arXiv.org.
    19. Jeonghwa Cha & Kyungbo Park & Hangook Kim & Jongyi Hong, 2023. "Crisis Index Prediction Based on Momentum Theory and Earnings Downside Risk Theory: Focusing on South Korea’s Energy Industry," Energies, MDPI, vol. 16(5), pages 1-20, February.
    20. Janusz Gajda & Rafał Walasek, 2020. "Fractional differentiation and its use in machine learning," Working Papers 2020-32, Faculty of Economic Sciences, University of Warsaw.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:jcsosc:v:7:y:2024:i:3:d:10.1007_s42001-024-00326-y. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.