IDEAS home Printed from https://ideas.repec.org/a/ebl/ecbull/eb-19-01019.html
   My bibliography  Save this article

Does Applying Deep Learning in Financial Sentiment Analysis Lead to Better Classification Performance?

Author

Listed:
  • Cuiyuan Wang

    (CUNY Graduate Center)

  • Tao Wang

    (Queens College and CUNY Graduate Center)

  • Changhe Yuan

    (Queens College and CUNY Graduate Center)

Abstract

Using a unique data set from Seeking Alpha, we compare the deep learning approach with traditional machine learning approaches in classifying financial text. We apply the long short-term memory (LSTM) as the deep learning method and Naive Bayes, SVM, Logistic Regression, XGBoost as the traditional machine learning approaches. The results suggest that the LSTM model outperforms the conventional machine learning methods on all metrics. Based on the t-SNE graph, the success of the LSTM model is partially explained as the high-accuracy LSTM model distinguishes between positive and negative important sentiment words while those words are chosen based on SHAP values and also appear in the widely used financial word dictionary, the Loughran-McDonald Dictionary (2011).

Suggested Citation

  • Cuiyuan Wang & Tao Wang & Changhe Yuan, 2020. "Does Applying Deep Learning in Financial Sentiment Analysis Lead to Better Classification Performance?," Economics Bulletin, AccessEcon, vol. 40(2), pages 1091-1105.
    • Handle: RePEc:ebl:ecbull:eb-19-01019
    as

    Download full text from publisher

    File URL: http://www.accessecon.com/Pubs/EB/2020/Volume40/EB-20-V40-I2-P94.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Tim Loughran & Bill Mcdonald, 2016. "Textual Analysis in Accounting and Finance: A Survey," Journal of Accounting Research, Wiley Blackwell, vol. 54(4), pages 1187-1230, September.
    2. Sanjiv R. Das & Mike Y. Chen, 2007. "Yahoo! for Amazon: Sentiment Extraction from Small Talk on the Web," Management Science, INFORMS, vol. 53(9), pages 1375-1388, September.
    3. Paul C. Tetlock, 2007. "Giving Content to Investor Sentiment: The Role of Media in the Stock Market," Journal of Finance, American Finance Association, vol. 62(3), pages 1139-1168, June.
    4. repec:bla:jfinan:v:59:y:2004:i:3:p:1259-1294 is not listed on IDEAS
    5. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    6. Hailiang Chen & Prabuddha De & Yu (Jeffrey) Hu & Byoung-Hyoun Hwang, 2014. "Wisdom of Crowds: The Value of Stock Opinions Transmitted Through Social Media," The Review of Financial Studies, Society for Financial Studies, vol. 27(5), pages 1367-1403.
    7. Feng Li, 2010. "The Information Content of Forward‐Looking Statements in Corporate Filings—A Naïve Bayesian Machine Learning Approach," Journal of Accounting Research, Wiley Blackwell, vol. 48(5), pages 1049-1102, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Jiao Ji & Oleksandr Talavera & Shuxing Yin, 2018. "The Hidden Information Content: Evidence from the Tone of Independent Director Reports," Working Papers 2018-28, Swansea University, School of Management.
    2. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    3. Loughran, Tim & McDonald, Bill & Pragidis, Ioannis, 2019. "Assimilation of oil news into prices," International Review of Financial Analysis, Elsevier, vol. 63(C), pages 105-118.
    4. Renault, Thomas, 2017. "Intraday online investor sentiment and return patterns in the U.S. stock market," Journal of Banking & Finance, Elsevier, vol. 84(C), pages 25-40.
    5. Christina Bannier & Thomas Pauls & Andreas Walter, 2019. "Content analysis of business communication: introducing a German dictionary," Journal of Business Economics, Springer, vol. 89(1), pages 79-123, February.
    6. Kothari, Pratik & Chance, Don M. & Ferris, Stephen P., 2021. "Bragging rights: Does corporate boasting imply value creation?," Journal of Corporate Finance, Elsevier, vol. 67(C).
    7. Qing Liu & Hosung Son, 2024. "Methods for aggregating investor sentiment from social media," Palgrave Communications, Palgrave Macmillan, vol. 11(1), pages 1-22, December.
    8. Shunyao (Cynthia) Jin, 2023. "More than words: Can tone of consumer product reviews help predict firms’ fundamentals?," Journal of Business Finance & Accounting, Wiley Blackwell, vol. 50(9-10), pages 1910-1942, October.
    9. Daniele Ballinari & Simon Behrendt, 2021. "How to gauge investor behavior? A comparison of online investor sentiment measures," Digital Finance, Springer, vol. 3(2), pages 169-204, June.
    10. Ingrid E. Fisher & Margaret R. Garnsey & Mark E. Hughes, 2016. "Natural Language Processing in Accounting, Auditing and Finance: A Synthesis of the Literature with a Roadmap for Future Research," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 23(3), pages 157-214, July.
    11. Anand, Abhinav & Basu, Sankarshan & Pathak, Jalaj & Thampy, Ashok, 2021. "The impact of sentiment on emerging stock markets," International Review of Economics & Finance, Elsevier, vol. 75(C), pages 161-177.
    12. Chau, Michael & Lin, Chih-Yung & Lin, Tse-Chun, 2020. "Wisdom of crowds before the 2007–2009 global financial crisis," Journal of Financial Stability, Elsevier, vol. 48(C).
    13. Enwei Zhu & Jing Wu & Hongyu Liu & Keyang Li, 2023. "A Sentiment Index of the Housing Market in China: Text Mining of Narratives on Social Media," The Journal of Real Estate Finance and Economics, Springer, vol. 66(1), pages 77-118, January.
    14. Thomas Renault, 2020. "Sentiment analysis and machine learning in finance: a comparison of methods and models on one million messages," Digital Finance, Springer, vol. 2(1), pages 1-13, September.
    15. Steven Heston & Nitish R. Sinha, 2016. "News versus Sentiment : Predicting Stock Returns from News Stories," Finance and Economics Discussion Series 2016-048, Board of Governors of the Federal Reserve System (U.S.).
    16. An, Suwei, 2023. "Essays on incentive contracts, M&As, and firm risk," Other publications TiSEM dd97d2f5-1c9d-47c5-ba62-f, Tilburg University, School of Economics and Management.
    17. Renato Camodeca & Alex Almici & Umberto Sagliaschi, 2018. "Sustainability Disclosure in Integrated Reporting: Does It Matter to Investors? A Cheap Talk Approach," Sustainability, MDPI, vol. 10(12), pages 1-34, November.
    18. Durnev, Art & Mangen, Claudine, 2020. "The spillover effects of MD&A disclosures for real investment: The role of industry competition," Journal of Accounting and Economics, Elsevier, vol. 70(1).
    19. Qingbin Meng & Congyi Ju & Qinghua Huang & Song Wang, 2023. "The informativeness of investor communication with corporate insiders: Evidence from China," International Finance, Wiley Blackwell, vol. 26(2), pages 189-207, August.
    20. Adam D. Nowak & Bradley S. Price & Patrick S. Smith, 2021. "Real Estate Dictionaries Across Space and Time," The Journal of Real Estate Finance and Economics, Springer, vol. 62(1), pages 139-163, January.

    More about this item

    Keywords

    Machine Learning; Deep Learning; Financial Social Media; Sentiment Analysis; Long Short-Term Memory;
    All these keywords.

    JEL classification:

    • C1 - Mathematical and Quantitative Methods - - Econometric and Statistical Methods and Methodology: General
    • G1 - Financial Economics - - General Financial Markets

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:ebl:ecbull:eb-19-01019. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: John P. Conley (email available below). General contact details of provider: .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.