IDEAS home Printed from https://ideas.repec.org/a/eee/finlet/v67y2024ipas1544612324008675.html
   My bibliography  Save this article

Unveiling tone manipulation in MD&A: Evidence from ChatGPT experiments

Author

Listed:
  • Song, Piaopeng
  • Lu, Hanglin
  • Zhang, Yongjie

Abstract

This research uses ChatGPT-3.5 and ChatGPT-4 to investigate tone manipulation in the management discussion and analysis of annual reports of Chinese-listed companies. We find that the quantification of emotional content in text using financial BERT model and dictionary approach is inconsistent due to two kinds of manipulation: "Expression Manipulation" and "Word Manipulation". Based on ChatGPT we verify the manipulative behavior of complex expressions and sentiment word substitutions. Our research suggests that using ChatGPT with appropriate cue words can help alleviate tone manipulation in financial texts, with GPT-4 having a better effect.

Suggested Citation

  • Song, Piaopeng & Lu, Hanglin & Zhang, Yongjie, 2024. "Unveiling tone manipulation in MD&A: Evidence from ChatGPT experiments," Finance Research Letters, Elsevier, vol. 67(PA).
    • Handle: RePEc:eee:finlet:v:67:y:2024:i:pa:s1544612324008675
    DOI: 10.1016/j.frl.2024.105837
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S1544612324008675
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.frl.2024.105837?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Mark H. Lang & Russell J. Lundholm, 2000. "Voluntary Disclosure and Equity Offerings: Reducing Information Asymmetry or Hyping the Stock?," Contemporary Accounting Research, John Wiley & Sons, vol. 17(4), pages 623-662, December.
    2. Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    3. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    4. Markus Leippold, 2023. "Sentiment Spin: Attacking Financial Sentiment with GPT-3," Swiss Finance Institute Research Paper Series 23-11, Swiss Finance Institute.
    5. Shuyu Zhang & Walter Aerts & Dunli Zhang & Zishan Chen, 2022. "Positive tone and initial coin offering," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 62(2), pages 2237-2266, June.
    6. Pekka Malo & Ankur Sinha & Pekka Korhonen & Jyrki Wallenius & Pyry Takala, 2014. "Good debt or bad debt: Detecting semantic orientations in economic texts," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 65(4), pages 782-796, April.
    7. Oehler, Andreas & Horn, Matthias, 2024. "Does ChatGPT provide better advice than robo-advisors?," Finance Research Letters, Elsevier, vol. 60(C).
    8. Wei, Tian & Wu, Han & Chu, Gang, 2023. "Is ChatGPT competent? Heterogeneity in the cognitive schemas of financial auditors and robots," International Review of Economics & Finance, Elsevier, vol. 88(C), pages 1389-1396.
    9. Tim Loughran & Bill Mcdonald, 2011. "When Is a Liability Not a Liability? Textual Analysis, Dictionaries, and 10‐Ks," Journal of Finance, American Finance Association, vol. 66(1), pages 35-65, February.
    10. Maciej P. Polak & Dane Morgan, 2024. "Extracting accurate materials data from research papers with conversational language models and prompt engineering," Nature Communications, Nature, vol. 15(1), pages 1-11, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    2. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    3. Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    4. Sui, Cong & Wang, Shuhan & Zheng, Wei, 2024. "Sentiment as a shipping market predictor: Testing market-specific language models," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 189(C).
    5. Kirtac, Kemal & Germano, Guido, 2024. "Sentiment trading with large language models," Finance Research Letters, Elsevier, vol. 62(PB).
    6. Chen, Cathy Yi-Hsuan & Fengler, Matthias R. & Härdle, Wolfgang Karl & Liu, Yanchu, 2022. "Media-expressed tone, option characteristics, and stock return predictability," Journal of Economic Dynamics and Control, Elsevier, vol. 134(C).
    7. Bingler, Julia Anna & Kraus, Mathias & Leippold, Markus & Webersinke, Nicolas, 2024. "How cheap talk in climate disclosures relates to climate initiatives, corporate emissions, and reputation risk," Journal of Banking & Finance, Elsevier, vol. 164(C).
    8. Allen H. Huang & Jianghua Shen & Amy Y. Zang, 2022. "The unintended benefit of the risk factor mandate of 2005," Review of Accounting Studies, Springer, vol. 27(4), pages 1319-1355, December.
    9. Moritz Scherrmann, 2023. "Multi-Label Topic Model for Financial Textual Data," Papers 2311.07598, arXiv.org.
    10. David M. Goldberg & Nohel Zaman & Arin Brahma & Mariano Aloiso, 2022. "Are mortgage loan closing delay risks predictable? A predictive analysis using text mining on discussion threads," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(3), pages 419-437, March.
    11. Alonso-Robisco, Andres & Carbó, José Manuel, 2023. "Analysis of CBDC narrative by central banks using large language models," Finance Research Letters, Elsevier, vol. 58(PC).
    12. Asier Guti'errez-Fandi~no & Miquel Noguer i Alonso & Petter Kolm & Jordi Armengol-Estap'e, 2021. "FinEAS: Financial Embedding Analysis of Sentiment," Papers 2111.00526, arXiv.org, revised Nov 2021.
    13. Ingrid E. Fisher & Margaret R. Garnsey & Mark E. Hughes, 2016. "Natural Language Processing in Accounting, Auditing and Finance: A Synthesis of the Literature with a Roadmap for Future Research," Intelligent Systems in Accounting, Finance and Management, John Wiley & Sons, Ltd., vol. 23(3), pages 157-214, July.
    14. Amel-Zadeh, Amir & Meeks, Geoff, 2019. "Bidder earnings forecasts in mergers and acquisitions," Journal of Corporate Finance, Elsevier, vol. 58(C), pages 373-392.
    15. Durnev, Art & Mangen, Claudine, 2020. "The spillover effects of MD&A disclosures for real investment: The role of industry competition," Journal of Accounting and Economics, Elsevier, vol. 70(1).
    16. Hsu, Audrey Wen-hsin & Wang, Tawei, 2013. "Does the market value corporate response to climate change?," Omega, Elsevier, vol. 41(2), pages 195-206.
    17. Johan, Sofia & Zhang, Yelin, 2020. "Quality revealing versus overstating in equity crowdfunding," Journal of Corporate Finance, Elsevier, vol. 65(C).
    18. Price, S. McKay & Doran, James S. & Peterson, David R. & Bliss, Barbara A., 2012. "Earnings conference calls and stock returns: The incremental informativeness of textual tone," Journal of Banking & Finance, Elsevier, vol. 36(4), pages 992-1011.
    19. De Franco, Gus & Shohfi, Thomas & Xu, Da & Zhu, Zhiwei (Vivi), 2023. "Fixed income conference calls," Journal of Accounting and Economics, Elsevier, vol. 75(1).
    20. Ankur Sinha & Satishwar Kedas & Rishu Kumar & Pekka Malo, 2022. "SEntFiN 1.0: Entity‐aware sentiment analysis for financial news," Journal of the Association for Information Science & Technology, Association for Information Science & Technology, vol. 73(9), pages 1314-1335, September.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:finlet:v:67:y:2024:i:pa:s1544612324008675. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/frl .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.