IDEAS home Printed from https://ideas.repec.org/a/gam/jijfss/v12y2024i3p60-d1423687.html
   My bibliography  Save this article

AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges

Author

Listed:
  • Li Xian Liu

    (College of Business, Law & Governance, James Cook University, 1 James Cook Drive, Douglas, QLD 4811, Australia)

  • Zhiyue Sun

    (School of Accounting, Economics & Finance, Curtin University, Kent Street, Perth, WA 6102, Australia)

  • Kunpeng Xu

    (School of Statistics and Information, Shanghai University of International Business and Economics, 1900 Wenxiang Rd, Songjiang District, Shanghai 201613, China)

  • Chao Chen

    (Accounting, Information System and Supply Chain, RMIT University, 124 La Trobe St, Melbourne, VIC 3000, Australia)

Abstract

The transformative impact of AI technologies on the financial sector has been a topic of increasing interest. This study investigates ChatGPT’s applications in financial reasoning and analysis and evaluates ChatGPT-4o’s effectiveness and limitations in conducting both basic and complex financial analysis tasks. By designing a series of multi-step, advanced reasoning tasks and establishing task-specific evaluation metrics, we assessed ChatGPT-4o’s performance compared to human analysts. Results indicate that while ChatGPT-4o demonstrates proficiency in basic and some complex financial tasks, it struggles with deep analytical and critical thinking tasks, especially in specialized finance areas. This study underscores the need for meticulous task formulation and robust evaluation in AI financial applications. While ChatGPT enhances efficiency, integrating it with human expertise is crucial for effective decision-making. Our findings highlight both the potential and limitations of ChatGPT-4o in financial analysis, providing valuable insights for future AI integration in the finance sector.

Suggested Citation

  • Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    • Handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7072/12/3/60/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7072/12/3/60/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    2. Hartmann, Jochen & Heitmann, Mark & Siebert, Christian & Schamp, Christina, 2023. "More than a Feeling: Accuracy and Application of Sentiment Analysis," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 75-87.
    3. Li Xian Liu & Shuangzhe Liu & Milind Sathye, 2021. "Predicting Bank Failures: A Synthesis of Literature and Directions for Future Research," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    4. Luiz Félix & Roman Kräussl & Philip Stork, 2020. "Implied volatility sentiment: a tale of two tails," Quantitative Finance, Taylor & Francis Journals, vol. 20(5), pages 823-849, May.
    5. Markus Leippold, 2023. "Sentiment Spin: Attacking Financial Sentiment with GPT-3," Swiss Finance Institute Research Paper Series 23-11, Swiss Finance Institute.
    6. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    7. Martin Jullum & Anders Løland & Ragnar Bang Huseby & Geir Ånonsen & Johannes Lorentzen, 2020. "Detecting money laundering transactions with machine learning," Journal of Money Laundering Control, Emerald Group Publishing Limited, vol. 23(1), pages 173-186, January.
    8. Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    2. Bingler, Julia Anna & Kraus, Mathias & Leippold, Markus & Webersinke, Nicolas, 2024. "How cheap talk in climate disclosures relates to climate initiatives, corporate emissions, and reputation risk," Journal of Banking & Finance, Elsevier, vol. 164(C).
    3. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    4. Rick Steinert & Saskia Altmann, 2023. "Linking microblogging sentiments to stock price movement: An application of GPT-4," Papers 2308.16771, arXiv.org.
    5. Stolowy, Hervé & Jeanjean, Thomas & Erkens, Michael, 2011. "The economic consequences of increasing the international visibility of financial reports," HEC Research Papers Series 957, HEC Paris.
    6. Yan Luo & Linying Zhou, 2020. "Textual tone in corporate financial disclosures: a survey of the literature," International Journal of Disclosure and Governance, Palgrave Macmillan, vol. 17(2), pages 101-110, September.
    7. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    8. Jascha-Alexander Koch & Michael Siering, 2019. "The recipe of successful crowdfunding campaigns," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(4), pages 661-679, December.
    9. Liu, Ya-Fei & Li, Hui & Liang, Sai, 2022. "Any reputation is a good reputation: influence of investor-perceived reputation in restructuring on hospitality firm performance," Annals of Tourism Research, Elsevier, vol. 92(C).
    10. Caporale, Guglielmo Maria & Spagnolo, Fabio & Spagnolo, Nicola, 2016. "Macro news and stock returns in the Euro area: A VAR-GARCH-in-mean analysis," International Review of Financial Analysis, Elsevier, vol. 45(C), pages 180-188.
    11. Altunbaş, Yener & Khan, Atiqur & Thornton, John, 2023. "Disclosure and bank risk: Evidence from European banks," Finance Research Letters, Elsevier, vol. 58(PB).
    12. Solomon, David H. & Soltes, Eugene & Sosyura, Denis, 2014. "Winners in the spotlight: Media coverage of fund holdings as a driver of flows," Journal of Financial Economics, Elsevier, vol. 113(1), pages 53-72.
    13. Ferdinand Graf, 2011. "Mechanically Extracted Company Signals and their Impact on Stock and Credit Markets," Working Paper Series of the Department of Economics, University of Konstanz 2011-18, Department of Economics, University of Konstanz.
    14. Xiaofei Zhao, 2017. "Does Information Intensity Matter for Stock Returns? Evidence from Form 8-K Filings," Management Science, INFORMS, vol. 63(5), pages 1382-1404, May.
    15. Thomas Renault, 2020. "Sentiment analysis and machine learning in finance: a comparison of methods and models on one million messages," Digital Finance, Springer, vol. 2(1), pages 1-13, September.
    16. Paul Hubert & Fabien Labondance, 2016. "Central Bank Sentiment and Policy Expectations," Working Papers hal-03459227, HAL.
    17. Lee A. Smales, 2016. "Time-varying relationship of news sentiment, implied volatility and stock returns," Applied Economics, Taylor & Francis Journals, vol. 48(51), pages 4942-4960, November.
    18. Lixing Mei & Yulei Rao & Mei Wang & Jianxin Wang, 2019. "Do investors post messages differently from mobile devices? The correlation between mobile Internet messages posting and stock returns," International Review of Economics, Springer;Happiness Economics and Interpersonal Relations (HEIRS), vol. 66(4), pages 423-452, December.
    19. Paul Brockman & Jim Cicon, 2013. "The Information Content Of Management Earnings Forecasts: An Analysis Of Hard Versus Soft Information," Journal of Financial Research, Southern Finance Association;Southwestern Finance Association, vol. 36(2), pages 147-174, June.
    20. Wu, Chunying & Xiong, Xiong & Gao, Ya, 2022. "The role of different information sources in information spread: Evidence from three media channels in China," International Review of Economics & Finance, Elsevier, vol. 80(C), pages 327-341.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.