IDEAS home Printed from https://ideas.repec.org/a/eee/finlet/v55y2023ipbs154461232300329x.html
   My bibliography  Save this article

Sentiment spin: Attacking financial sentiment with GPT-3

Author

Listed:
  • Leippold, Markus

Abstract

In this study, we explore the susceptibility of financial sentiment analysis to adversarial attacks that manipulate financial texts. With the rise of AI readership in the financial sector, companies are adapting their language and disclosures to fit AI processing better, leading to concerns about the potential for manipulation. In the finance literature, keyword-based methods, such as dictionaries, are still widely used for financial sentiment analysis due to their perceived transparency. However, our research demonstrates the vulnerability of keyword-based approaches by successfully generating adversarial attacks using the sophisticated transformer model, GPT-3. With a success rate of nearly 99% for negative sentences in the Financial Phrase Bank, a widely used database for financial sentiment analysis, we highlight the importance of incorporating robust methods, such as context-aware approaches such as BERT, in financial sentiment analysis.

Suggested Citation

  • Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    • Handle: RePEc:eee:finlet:v:55:y:2023:i:pb:s154461232300329x
    DOI: 10.1016/j.frl.2023.103957
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S154461232300329X
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.frl.2023.103957?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Saeid Vaghefi & Qian Wang & Veruska Muccione & Jingwei Ni & Mathias Kraus & Julia Bingler & Tobias Schimanski & Chiara Colesanti Senni & Nicolas Webersinke & Christian Huggel & Markus Leippold, 2023. "ChatClimate: Grounding Conversational AI in Climate Science," Swiss Finance Institute Research Paper Series 23-88, Swiss Finance Institute.
    2. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    3. Pekka Malo & Ankur Sinha & Pekka Korhonen & Jyrki Wallenius & Pyry Takala, 2014. "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 782-796, April.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Rick Steinert & Saskia Altmann, 2023. "Linking microblogging sentiments to stock price movement: An application of GPT-4," Papers 2308.16771, arXiv.org.
    2. Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    3. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    4. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," Finance Research Letters, Elsevier, vol. 62(PB).
    2. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    3. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," LSE Research Online Documents on Economics 122592, London School of Economics and Political Science, LSE Library.
    4. Ankur Sinha & Satishwar Kedas & Rishu Kumar & Pekka Malo, 2022. "SEntFiN 1.0: Entity‐aware sentiment analysis for financial news," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1314-1335, September.
    5. Abdollahi, Hooman & Fjesme, Sturla L. & Sirnes, Espen, 2024. "Measuring market volatility connectedness to media sentiment," The North American Journal of Economics and Finance, Elsevier, vol. 71(C).
    6. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    7. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2018. "Textual Sentiment, Option Characteristics, and Stock Return Predictability," IRTG 1792 Discussion Papers 2018-023, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    8. Agam Shah & Sudheer Chava, 2023. "Zero is Not Hero Yet: Benchmarking Zero-Shot Performance of LLMs for Financial Tasks," Papers 2305.16633, arXiv.org.
    9. Runmei Luo & Yong Ye, 2024. "Pressure from words: The tone of investors in Chinese earnings communication conferences and managerial myopia," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 64(1), pages 833-868, March.
    10. Moritz Scherrmann, 2023. "Multi-Label Topic Model for Financial Textual Data," Papers 2311.07598, arXiv.org.
    11. David M. Goldberg & Nohel Zaman & Arin Brahma & Mariano Aloiso, 2022. "Are mortgage loan closing delay risks predictable? A predictive analysis using text mining on discussion threads," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(3), pages 419-437, March.
    12. Asier Guti'errez-Fandi~no & Miquel Noguer i Alonso & Petter Kolm & Jordi Armengol-Estap'e, 2021. "FinEAS: Financial Embedding Analysis of Sentiment," Papers 2111.00526, arXiv.org, revised Nov 2021.
    13. Ingrid E. Fisher & Margaret R. Garnsey & Mark E. Hughes, 2016. "Natural Language Processing in Accounting, Auditing and Finance: A Synthesis of the Literature with a Roadmap for Future Research," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 23(3), pages 157-214, July.
    14. Jozef Barunik & Cathy Yi-Hsuan Chen & Jan Vecer, 2019. "Sentiment-Driven Stochastic Volatility Model: A High-Frequency Textual Tool for Economists," Papers 1906.00059, arXiv.org.
    15. Boyu Zhang & Hongyang Yang & Xiao-Yang Liu, 2023. "Instruct-FinGPT: Financial Sentiment Analysis by Instruction Tuning of General-Purpose Large Language Models," Papers 2306.12659, arXiv.org.
    16. Bommes, Elisabeth & Chen, Cathy Yi-Hsuan & Härdle, Wolfgang Karl, 2018. "Textual Sentiment and Sector specific reaction," IRTG 1792 Discussion Papers 2018-043, Humboldt University of Berlin, International Research Training Group 1792 "High Dimensional Nonstationary Time Series".
    17. Wehrheim, Lino, 2021. "The sound of silence: On the (in)visibility of economists in the media," Working Papers 30, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    18. Gu, Chen & Kurov, Alexander & Wolfe, Marketa Halova, 2018. "Relief Rallies after FOMC Announcements as a Resolution of Uncertainty," Journal of Empirical Finance, Elsevier, vol. 49(C), pages 1-18.
    19. Müller, Karsten, 2020. "German forecasters' narratives: How informative are German business cycle forecast reports?," Working Papers 23, German Research Foundation's Priority Programme 1859 "Experience and Expectation. Historical Foundations of Economic Behaviour", Humboldt University Berlin.
    20. Aaryan Gupta & Vinya Dengre & Hamza Abubakar Kheruwala & Manan Shah, 2020. "Comprehensive review of text-mining applications in finance," Financial Innovation, Springer;Southwestern University of Finance and Economics, vol. 6(1), pages 1-25, December.

    More about this item

    Keywords

    sentiment analysis in financial markets; Keyword-based approach; FinBERT; GPT-3;
    All these keywords.

    JEL classification:

    • G2 - Financial Economics - - Financial Institutions and Services
    • G38 - Financial Economics - - Corporate Finance and Governance - - - Government Policy and Regulation
    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • M48 - Business Administration and Business Economics; Marketing; Accounting; Personnel Economics - - Accounting - - - Government Policy and Regulation

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finlet:v:55:y:2023:i:pb:s154461232300329x. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/frl .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.