IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2409.11540.html
   My bibliography  Save this paper

What Does ChatGPT Make of Historical Stock Returns? Extrapolation and Miscalibration in LLM Stock Return Forecasts

Author

Listed:
  • Shuaiyu Chen
  • T. Clifton Green
  • Huseyin Gulen
  • Dexin Zhou

Abstract

We examine how large language models (LLMs) interpret historical stock returns and compare their forecasts with estimates from a crowd-sourced platform for ranking stocks. While stock returns exhibit short-term reversals, LLM forecasts over-extrapolate, placing excessive weight on recent performance similar to humans. LLM forecasts appear optimistic relative to historical and future realized returns. When prompted for 80% confidence interval predictions, LLM responses are better calibrated than survey evidence but are pessimistic about outliers, leading to skewed forecast distributions. The findings suggest LLMs manifest common behavioral biases when forecasting expected returns but are better at gauging risks than humans.

Suggested Citation

  • Shuaiyu Chen & T. Clifton Green & Huseyin Gulen & Dexin Zhou, 2024. "What Does ChatGPT Make of Historical Stock Returns? Extrapolation and Miscalibration in LLM Stock Return Forecasts," Papers 2409.11540, arXiv.org.
    • Handle: RePEc:arx:papers:2409.11540
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2409.11540
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Andreas Fuster & Paul Goldsmith‐Pinkham & Tarun Ramadorai & Ansgar Walther, 2022. "Predictably Unequal? The Effects of Machine Learning on Credit Markets," Journal of Finance, American Finance Association, vol. 77(1), pages 5-47, February.
    2. Theresa Kuchler & Basit Zafar, 2019. "Personal Experiences and Expectations about Aggregate Outcomes," Journal of Finance, American Finance Association, vol. 74(5), pages 2491-2542, October.
    3. García, Diego & Hu, Xiaowen & Rohrer, Maximilian, 2023. "The colour of finance words," Journal of Financial Economics, Elsevier, vol. 147(3), pages 525-549.
    4. Chen, Hsiu-Lang & Jegadeesh, Narasimhan & Wermers, Russ, 2000. "The Value of Active Mutual Fund Management: An Examination of the Stockholdings and Trades of Fund Managers," Journal of Financial and Quantitative Analysis, Cambridge University Press, vol. 35(3), pages 343-368, September.
    5. Bartlett, Robert & Morse, Adair & Stanton, Richard & Wallace, Nancy, 2022. "Consumer-lending discrimination in the FinTech Era," Journal of Financial Economics, Elsevier, vol. 143(1), pages 30-56.
    6. Alex Kim & Maximilian Muhn & Valeri Nikolaev, 2023. "Bloated Disclosures: Can ChatGPT Help Investors Process Information?," Papers 2306.10224, arXiv.org, revised Feb 2024.
    7. Camelia M. Kuhnen, 2015. "Asymmetric Learning from Financial Information," Journal of Finance, American Finance Association, vol. 70(5), pages 2029-2062, October.
    8. Azi Ben-Rephael & Zhi Da & Ryan D. Israelsen, 2017. "It Depends on Where You Search: Institutional Investor Attention and Underreaction to News," The Review of Financial Studies, Society for Financial Studies, vol. 30(9), pages 3009-3047.
    9. Bruce N. Lehmann, 1990. "Fads, Martingales, and Market Efficiency," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 105(1), pages 1-28.
    10. Anastassia Fedyk & James Hodson & Natalya Khimich & Tatiana Fedyk, 2022. "Is artificial intelligence improving the audit process?," Review of Accounting Studies, Springer, vol. 27(3), pages 938-985, September.
    11. Stefano Cassella & Huseyin Gulen, 2018. "Extrapolation Bias and the Predictability of Stock Returns by Price-Scaled Variables," The Review of Financial Studies, Society for Financial Studies, vol. 31(11), pages 4345-4397.
    12. Doron Avramov & Tarun Chordia & Amit Goyal, 2006. "Liquidity and Autocorrelations in Individual Stock Returns," Journal of Finance, American Finance Association, vol. 61(5), pages 2365-2394, October.
    13. Barberis, Nicholas & Greenwood, Robin & Jin, Lawrence & Shleifer, Andrei, 2015. "X-CAPM: An extrapolative capital asset pricing model," Journal of Financial Economics, Elsevier, vol. 115(1), pages 1-24.
    14. Paul Glasserman & Caden Lin, 2023. "Assessing Look-Ahead Bias in Stock Return Predictions Generated By GPT Sentiment Analysis," Papers 2309.17322, arXiv.org.
    15. Peiyao Li & Noah Castelo & Zsolt Katona & Miklos Sarvary, 2024. "Frontiers: Determining the Validity of Large Language Models for Automated Perceptual Analysis," Marketing Science, INFORMS, vol. 43(2), pages 254-266, March.
    16. Reher, Michael & Sokolinski, Stanislav, 2024. "Robo advisors and access to wealth management," Journal of Financial Economics, Elsevier, vol. 155(C).
    17. Jegadeesh, Narasimhan, 1990. "Evidence of Predictable Behavior of Security Returns," Journal of Finance, American Finance Association, vol. 45(3), pages 881-898, July.
    18. Daniel, Kent, et al, 1997. "Measuring Mutual Fund Performance with Characteristic-Based Benchmarks," Journal of Finance, American Finance Association, vol. 52(3), pages 1035-1058, July.
    19. Obaid, Khaled & Pukthuanthong, Kuntara, 2022. "A picture is worth a thousand words: Measuring investor sentiment by combining machine learning and photos from news," Journal of Financial Economics, Elsevier, vol. 144(1), pages 273-297.
    20. Cao, Sean & Jiang, Wei & Wang, Junbo & Yang, Baozhong, 2024. "From Man vs. Machine to Man + Machine: The art and AI of stock analyses," Journal of Financial Economics, Elsevier, vol. 160(C).
    21. Da, Zhi & Huang, Xing & Jin, Lawrence J., 2021. "Extrapolative beliefs in the cross-section: What can we learn from the crowds?," Journal of Financial Economics, Elsevier, vol. 140(1), pages 175-196.
    22. Zhi Da & Qianqiu Liu & Ernst Schaumburg, 2014. "A Closer Look at the Short-Term Return Reversal," Management Science, INFORMS, vol. 60(3), pages 658-674, March.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Da, Zhi & Huang, Xing & Jin, Lawrence J., 2021. "Extrapolative beliefs in the cross-section: What can we learn from the crowds?," Journal of Financial Economics, Elsevier, vol. 140(1), pages 175-196.
    2. Cakici, Nusret & Zaremba, Adam, 2023. "Recency bias and the cross-section of international stock returns," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 84(C).
    3. Cakici, Nusret & Zaremba, Adam, 2022. "Salience theory and the cross-section of stock returns: International and further evidence," Journal of Financial Economics, Elsevier, vol. 146(2), pages 689-725.
    4. Mohrschladt, Hannes, 2021. "The ordering of historical returns and the cross-section of subsequent returns," Journal of Banking & Finance, Elsevier, vol. 125(C).
    5. Adam Zaremba & Jacob Koby Shemer, 2018. "Price-Based Investment Strategies," Springer Books, Springer, number 978-3-319-91530-2, December.
    6. Kang, Junqing & Lin, Shen & Xiong, Xiong, 2022. "What drives intraday reversal? illiquidity or liquidity oversupply?," Journal of Economic Dynamics and Control, Elsevier, vol. 136(C).
    7. Gong, Qiang & Liu, Ming & Liu, Qianqiu, 2015. "Momentum is really short-term momentum," Journal of Banking & Finance, Elsevier, vol. 50(C), pages 169-182.
    8. Dyl, Edward A. & Yuksel, H. Zafer & Zaynutdinova, Gulnara R., 2019. "Price reversals and price continuations following large price movements," Journal of Business Research, Elsevier, vol. 95(C), pages 1-12.
    9. Goh, Jihoon & Jeong, Giho & Kang, Jangkoo, 2022. "The reference dependency of short-term reversal," International Review of Economics & Finance, Elsevier, vol. 78(C), pages 195-211.
    10. Russell Jame, 2018. "Liquidity Provision and the Cross Section of Hedge Fund Returns," Management Science, INFORMS, vol. 64(7), pages 3288-3312, July.
    11. Jiang, George J. & Zhu, Kevin X., 2017. "Information Shocks and Short-Term Market Underreaction," Journal of Financial Economics, Elsevier, vol. 124(1), pages 43-64.
    12. Kang, Moonsoo & Khaksari, S. & Nam, Kiseok, 2018. "Corporate investment, short-term return reversal, and stock liquidity," Journal of Financial Markets, Elsevier, vol. 39(C), pages 68-83.
    13. Oesinghaus, Andreas, 2024. "Analysts’ extrapolative expectations in the cross-section," Journal of Economics and Business, Elsevier, vol. 130(C).
    14. Zhaobo Zhu & Licheng Sun, 2024. "Economic policy uncertainty and short-term reversals," Post-Print hal-04691597, HAL.
    15. Jin, Lawrence J. & Sui, Pengfei, 2022. "Asset pricing with return extrapolation," Journal of Financial Economics, Elsevier, vol. 145(2), pages 273-295.
    16. Bianchi, Daniele & Babiak, Mykola & Dickerson, Alexander, 2022. "Trading volume and liquidity provision in cryptocurrency markets," Journal of Banking & Finance, Elsevier, vol. 142(C).
    17. Kotaro Miwa, 2019. "Short-Term Return Reversals and Intraday Transactions," Quarterly Journal of Finance (QJF), World Scientific Publishing Co. Pte. Ltd., vol. 9(01), pages 1-25, March.
    18. Isaenko, Sergey, 2023. "Trading strategies and the frequency of time-series," The Quarterly Review of Economics and Finance, Elsevier, vol. 90(C), pages 267-283.
    19. Anh Duy Nguyen, 2020. "Alternative reversal variable," Post-Print hal-02388743, HAL.
    20. Steven L. Heston & Robert A. Korajczyk & Ronnie Sadka, 2010. "Intraday Patterns in the Cross‐section of Stock Returns," Journal of Finance, American Finance Association, vol. 65(4), pages 1369-1407, August.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2409.11540. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.