IDEAS home Printed from https://ideas.repec.org/a/gam/jijfss/v12y2024i3p60-d1423687.html
   My bibliography  Save this article

AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges

Author

Listed:
  • Li Xian Liu

    (College of Business, Law & Governance, James Cook University, 1 James Cook Drive, Douglas, QLD 4811, Australia)

  • Zhiyue Sun

    (School of Accounting, Economics & Finance, Curtin University, Kent Street, Perth, WA 6102, Australia)

  • Kunpeng Xu

    (School of Statistics and Information, Shanghai University of International Business and Economics, 1900 Wenxiang Rd, Songjiang District, Shanghai 201613, China)

  • Chao Chen

    (Accounting, Information System and Supply Chain, RMIT University, 124 La Trobe St, Melbourne, VIC 3000, Australia)

Abstract

The transformative impact of AI technologies on the financial sector has been a topic of increasing interest. This study investigates ChatGPT’s applications in financial reasoning and analysis and evaluates ChatGPT-4o’s effectiveness and limitations in conducting both basic and complex financial analysis tasks. By designing a series of multi-step, advanced reasoning tasks and establishing task-specific evaluation metrics, we assessed ChatGPT-4o’s performance compared to human analysts. Results indicate that while ChatGPT-4o demonstrates proficiency in basic and some complex financial tasks, it struggles with deep analytical and critical thinking tasks, especially in specialized finance areas. This study underscores the need for meticulous task formulation and robust evaluation in AI financial applications. While ChatGPT enhances efficiency, integrating it with human expertise is crucial for effective decision-making. Our findings highlight both the potential and limitations of ChatGPT-4o in financial analysis, providing valuable insights for future AI integration in the finance sector.

Suggested Citation

  • Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    • Handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7072/12/3/60/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7072/12/3/60/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Martin Jullum & Anders Løland & Ragnar Bang Huseby & Geir Ånonsen & Johannes Lorentzen, 2020. "Detecting money laundering transactions with machine learning," Journal of Money Laundering Control, Emerald Group Publishing Limited, vol. 23(1), pages 173-186, January.
    2. Luiz Félix & Roman Kräussl & Philip Stork, 2020. "Implied volatility sentiment: a tale of two tails," Quantitative Finance, Taylor & Francis Journals, vol. 20(5), pages 823-849, May.
    3. Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    4. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    5. Markus Leippold, 2023. "Sentiment Spin: Attacking Financial Sentiment with GPT-3," Swiss Finance Institute Research Paper Series 23-11, Swiss Finance Institute.
    6. Hartmann, Jochen & Heitmann, Mark & Siebert, Christian & Schamp, Christina, 2023. "More than a Feeling: Accuracy and Application of Sentiment Analysis," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 75-87.
    7. Li Xian Liu & Shuangzhe Liu & Milind Sathye, 2021. "Predicting Bank Failures: A Synthesis of Literature and Directions for Future Research," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    8. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    2. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    3. Rick Steinert & Saskia Altmann, 2023. "Linking microblogging sentiments to stock price movement: An application of GPT-4," Papers 2308.16771, arXiv.org.
    4. Stolowy, Hervé & Jeanjean, Thomas & Erkens, Michael, 2011. "The economic consequences of increasing the international visibility of financial reports," HEC Research Papers Series 957, HEC Paris.
    5. David E. Allen & Michael McAleer & Abhay K. Singh, 2019. "Daily market news sentiment and stock prices," Applied Economics, Taylor & Francis Journals, vol. 51(30), pages 3212-3235, June.
    6. Yousaf, Imran & Youssef, Manel & Goodell, John W., 2022. "Quantile connectedness between sentiment and financial markets: Evidence from the S&P 500 twitter sentiment index," International Review of Financial Analysis, Elsevier, vol. 83(C).
    7. Yan Luo & Linying Zhou, 2020. "Textual tone in corporate financial disclosures: a survey of the literature," International Journal of Disclosure and Governance, Palgrave Macmillan, vol. 17(2), pages 101-110, September.
    8. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    9. Jascha-Alexander Koch & Michael Siering, 2019. "The recipe of successful crowdfunding campaigns," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(4), pages 661-679, December.
    10. Liu, Ya-Fei & Li, Hui & Liang, Sai, 2022. "Any reputation is a good reputation: influence of investor-perceived reputation in restructuring on hospitality firm performance," Annals of Tourism Research, Elsevier, vol. 92(C).
    11. Michael J. Seiler & David M. Harrison, 2011. "Perceived Versus Actual Susceptibility to Normative Influence in the Presence of Defaulting Landlords," Review of Behavioral Finance, Emerald Group Publishing Limited, vol. 3(2), pages 55-77, September.
    12. Caporale, Guglielmo Maria & Spagnolo, Fabio & Spagnolo, Nicola, 2016. "Macro news and stock returns in the Euro area: A VAR-GARCH-in-mean analysis," International Review of Financial Analysis, Elsevier, vol. 45(C), pages 180-188.
    13. Leilane de Freitas Rocha Cambara & Roberto Meurer, 2023. "News sentiment and foreign portfolio investment in Brazil," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(3), pages 3332-3348, July.
    14. Rice, Stephen & Crouse, Sean R. & Winter, Scott R. & Rice, Connor, 2024. "The advantages and limitations of using ChatGPT to enhance technological research," Technology in Society, Elsevier, vol. 76(C).
    15. repec:hal:spmain:info:hdl:2441/3mgbd73vkp9f9oje7utooe7vpg is not listed on IDEAS
    16. Yang-Cheng Lu & Yu-Chen Wei, 2013. "The Chinese News Sentiment around Earnings Announcements," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(3), pages 44-58, October.
    17. Ahmed, Yousry & Elshandidy, Tamer, 2016. "The effect of bidder conservatism on M&A decisions: Text-based evidence from US 10-K filings," International Review of Financial Analysis, Elsevier, vol. 46(C), pages 176-190.
    18. Altunbaş, Yener & Khan, Atiqur & Thornton, John, 2023. "Disclosure and bank risk: Evidence from European banks," Finance Research Letters, Elsevier, vol. 58(PB).
    19. Liebmann, Michael & Orlov, Alexei G. & Neumann, Dirk, 2016. "The tone of financial news and the perceptions of stock and CDS traders," International Review of Financial Analysis, Elsevier, vol. 46(C), pages 159-175.
    20. Solomon, David H. & Soltes, Eugene & Sosyura, Denis, 2014. "Winners in the spotlight: Media coverage of fund holdings as a driver of flows," Journal of Financial Economics, Elsevier, vol. 113(1), pages 53-72.
    21. Ferdinand Graf, 2011. "Mechanically Extracted Company Signals and their Impact on Stock and Credit Markets," Working Paper Series of the Department of Economics, University of Konstanz 2011-18, Department of Economics, University of Konstanz.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.