IDEAS home Printed from https://ideas.repec.org/a/gam/jijfss/v12y2024i3p60-d1423687.html
   My bibliography  Save this article

AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges

Author

Listed:
  • Li Xian Liu

    (College of Business, Law & Governance, James Cook University, 1 James Cook Drive, Douglas, QLD 4811, Australia)

  • Zhiyue Sun

    (School of Accounting, Economics & Finance, Curtin University, Kent Street, Perth, WA 6102, Australia)

  • Kunpeng Xu

    (School of Statistics and Information, Shanghai University of International Business and Economics, 1900 Wenxiang Rd, Songjiang District, Shanghai 201613, China)

  • Chao Chen

    (Accounting, Information System and Supply Chain, RMIT University, 124 La Trobe St, Melbourne, VIC 3000, Australia)

Abstract

The transformative impact of AI technologies on the financial sector has been a topic of increasing interest. This study investigates ChatGPT’s applications in financial reasoning and analysis and evaluates ChatGPT-4o’s effectiveness and limitations in conducting both basic and complex financial analysis tasks. By designing a series of multi-step, advanced reasoning tasks and establishing task-specific evaluation metrics, we assessed ChatGPT-4o’s performance compared to human analysts. Results indicate that while ChatGPT-4o demonstrates proficiency in basic and some complex financial tasks, it struggles with deep analytical and critical thinking tasks, especially in specialized finance areas. This study underscores the need for meticulous task formulation and robust evaluation in AI financial applications. While ChatGPT enhances efficiency, integrating it with human expertise is crucial for effective decision-making. Our findings highlight both the potential and limitations of ChatGPT-4o in financial analysis, providing valuable insights for future AI integration in the finance sector.

Suggested Citation

  • Li Xian Liu & Zhiyue Sun & Kunpeng Xu & Chao Chen, 2024. "AI-Driven Financial Analysis: Exploring ChatGPT’s Capabilities and Challenges," IJFS, MDPI, vol. 12(3), pages 1-35, June.
    • Handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/2227-7072/12/3/60/pdf
    Download Restriction: no
    File URL: https://www.mdpi.com/2227-7072/12/3/60/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Luiz Félix & Roman Kräussl & Philip Stork, 2020. "Implied volatility sentiment: a tale of two tails," Quantitative Finance, Taylor & Francis Journals, vol. 20(5), pages 823-849, May.
    2. Markus Leippold, 2023. "Sentiment Spin: Attacking Financial Sentiment with GPT-3," Swiss Finance Institute Research Paper Series 23-11, Swiss Finance Institute.
    3. Paul C. Tetlock & Maytal Saar‐Tsechansky & Sofus Macskassy, 2008. "More Than Words: Quantifying Language to Measure Firms' Fundamentals," Journal of Finance, American Finance Association, vol. 63(3), pages 1437-1467, June.
    4. Martin Jullum & Anders Løland & Ragnar Bang Huseby & Geir Ånonsen & Johannes Lorentzen, 2020. "Detecting money laundering transactions with machine learning," Journal of Money Laundering Control, Emerald Group Publishing Limited, vol. 23(1), pages 173-186, January.
    5. Leippold, Markus, 2023. "Sentiment spin: Attacking financial sentiment with GPT-3," Finance Research Letters, Elsevier, vol. 55(PB).
    6. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    7. Hartmann, Jochen & Heitmann, Mark & Siebert, Christian & Schamp, Christina, 2023. "More than a Feeling: Accuracy and Application of Sentiment Analysis," International Journal of Research in Marketing, Elsevier, vol. 40(1), pages 75-87.
    8. Li Xian Liu & Shuangzhe Liu & Milind Sathye, 2021. "Predicting Bank Failures: A Synthesis of Literature and Directions for Future Research," JRFM, MDPI, vol. 14(10), pages 1-24, October.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Smales, Lee A., 2023. "Classification of RBA monetary policy announcements using ChatGPT," Finance Research Letters, Elsevier, vol. 58(PC).
    2. Sui, Cong & Wang, Shuhan & Zheng, Wei, 2024. "Sentiment as a shipping market predictor: Testing market-specific language models," Transportation Research Part E: Logistics and Transportation Review, Elsevier, vol. 189(C).
    3. Song, Piaopeng & Lu, Hanglin & Zhang, Yongjie, 2024. "Unveiling tone manipulation in MD&A: Evidence from ChatGPT experiments," Finance Research Letters, Elsevier, vol. 67(PA).
    4. Bingler, Julia Anna & Kraus, Mathias & Leippold, Markus & Webersinke, Nicolas, 2024. "How cheap talk in climate disclosures relates to climate initiatives, corporate emissions, and reputation risk," Journal of Banking & Finance, Elsevier, vol. 164(C).
    5. Dong, Mengming Michael & Stratopoulos, Theophanis C. & Wang, Victor Xiaoqi, 2024. "A scoping review of ChatGPT research in accounting and finance," International Journal of Accounting Information Systems, Elsevier, vol. 55(C).
    6. Olamilekan Shobayo & Sidikat Adeyemi-Longe & Olusogo Popoola & Bayode Ogunleye, 2024. "Innovative Sentiment Analysis and Prediction of Stock Price Using FinBERT, GPT-4 and Logistic Regression: A Data-Driven Approach," Papers 2412.06837, arXiv.org.
    7. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.
    8. Rick Steinert & Saskia Altmann, 2023. "Linking microblogging sentiments to stock price movement: An application of GPT-4," Papers 2308.16771, arXiv.org.
    9. Stolowy, Hervé & Jeanjean, Thomas & Erkens, Michael, 2011. "The economic consequences of increasing the international visibility of financial reports," HEC Research Papers Series 957, HEC Paris.
    10. David E. Allen & Michael McAleer & Abhay K. Singh, 2019. "Daily market news sentiment and stock prices," Applied Economics, Taylor & Francis Journals, vol. 51(30), pages 3212-3235, June.
    11. Yousaf, Imran & Youssef, Manel & Goodell, John W., 2022. "Quantile connectedness between sentiment and financial markets: Evidence from the S&P 500 twitter sentiment index," International Review of Financial Analysis, Elsevier, vol. 83(C).
    12. Yan Luo & Linying Zhou, 2020. "Textual tone in corporate financial disclosures: a survey of the literature," International Journal of Disclosure and Governance, Palgrave Macmillan, vol. 17(2), pages 101-110, September.
    13. Kamaladdin Fataliyev & Aneesh Chivukula & Mukesh Prasad & Wei Liu, 2021. "Stock Market Analysis with Text Data: A Review," Papers 2106.12985, arXiv.org, revised Jul 2021.
    14. Jascha-Alexander Koch & Michael Siering, 2019. "The recipe of successful crowdfunding campaigns," Electronic Markets, Springer;IIM University of St. Gallen, vol. 29(4), pages 661-679, December.
    15. Liu, Ya-Fei & Li, Hui & Liang, Sai, 2022. "Any reputation is a good reputation: influence of investor-perceived reputation in restructuring on hospitality firm performance," Annals of Tourism Research, Elsevier, vol. 92(C).
    16. Jan Ole Krugmann & Jochen Hartmann, 2024. "Sentiment Analysis in the Age of Generative AI," Customer Needs and Solutions, Springer;Institute for Sustainable Innovation and Growth (iSIG), vol. 11(1), pages 1-19, December.
    17. Eryka Probierz & Adam Galuszka & Katarzyna Klimczak & Karol Jedrasiak & Tomasz Wisniewski & Tomasz Dzida, 2021. "Financial Sentiment on Twitter's Community and it's Connection to Polish Stock Market Movements in Context of Behavior Modelling," European Research Studies Journal, European Research Studies Journal, vol. 0(4 - Part ), pages 56-65.
    18. Michael J. Seiler & David M. Harrison, 2011. "Perceived Versus Actual Susceptibility to Normative Influence in the Presence of Defaulting Landlords," Review of Behavioral Finance, Emerald Group Publishing Limited, vol. 3(2), pages 55-77, September.
    19. Caporale, Guglielmo Maria & Spagnolo, Fabio & Spagnolo, Nicola, 2016. "Macro news and stock returns in the Euro area: A VAR-GARCH-in-mean analysis," International Review of Financial Analysis, Elsevier, vol. 45(C), pages 180-188.
    20. Leilane de Freitas Rocha Cambara & Roberto Meurer, 2023. "News sentiment and foreign portfolio investment in Brazil," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 28(3), pages 3332-3348, July.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijfss:v:12:y:2024:i:3:p:60-:d:1423687. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.