IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2309.00649.html
   My bibliography  Save this paper

GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice

Author

Listed:
  • Pawe{l} Niszczota
  • Sami Abbas

Abstract

We assess the ability of GPT -- a large language model -- to serve as a financial robo-advisor for the masses, by using a financial literacy test. Davinci and ChatGPT based on GPT-3.5 score 66% and 65% on the financial literacy test, respectively, compared to a baseline of 33%. However, ChatGPT based on GPT-4 achieves a near-perfect 99% score, pointing to financial literacy becoming an emergent ability of state-of-the-art models. We use the Judge-Advisor System and a savings dilemma to illustrate how researchers might assess advice-utilization from large language models. We also present a number of directions for future research.

Suggested Citation

  • Pawe{l} Niszczota & Sami Abbas, 2023. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice," Papers 2309.00649, arXiv.org, revised Sep 2024.
    • Handle: RePEc:arx:papers:2309.00649
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2309.00649
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. van Rooij, Maarten & Lusardi, Annamaria & Alessie, Rob, 2011. "Financial literacy and stock market participation," Journal of Financial Economics, Elsevier, vol. 101(2), pages 449-472, August.
    2. Sniezek, Janet A. & Buckley, Timothy, 1995. "Cueing and Cognitive Conflict in Judge-Advisor Decision Making," Organizational Behavior and Human Decision Processes, Elsevier, vol. 62(2), pages 159-174, May.
    3. Dowling, Michael & Lucey, Brian, 2023. "ChatGPT for (Finance) research: The Bananarama Conjecture," Finance Research Letters, Elsevier, vol. 53(C).
    4. David R. Lewis, 2018. "The perils of overconfidence: Why many consumers fail to seek advice when they really should," Journal of Financial Services Marketing, Palgrave Macmillan, vol. 23(2), pages 104-111, June.
    5. Palan, Stefan & Schitter, Christian, 2018. "Prolific.ac—A subject pool for online experiments," Journal of Behavioral and Experimental Finance, Elsevier, vol. 17(C), pages 22-27.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Pelster, Matthias & Val, Joel, 2024. "Can ChatGPT assist in picking stocks?," Finance Research Letters, Elsevier, vol. 59(C).
    2. Oehler, Andreas & Horn, Matthias, 2024. "Does ChatGPT provide better advice than robo-advisors?," Finance Research Letters, Elsevier, vol. 60(C).
    3. Jeongbin Kim & Matthew Kovach & Kyu-Min Lee & Euncheol Shin & Hector Tzavellas, 2024. "Learning to be Homo Economicus: Can an LLM Learn Preferences from Choice," Papers 2401.07345, arXiv.org.
    4. Ko, Hyungjin & Lee, Jaewook, 2024. "Can ChatGPT improve investment decisions? From a portfolio management perspective," Finance Research Letters, Elsevier, vol. 64(C).
    5. Yuqi Nie & Yaxuan Kong & Xiaowen Dong & John M. Mulvey & H. Vincent Poor & Qingsong Wen & Stefan Zohren, 2024. "A Survey of Large Language Models for Financial Applications: Progress, Prospects and Challenges," Papers 2406.11903, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Niszczota, Paweł & Abbas, Sami, 2023. "GPT has become financially literate: Insights from financial literacy tests of GPT and a preliminary test of how people use it as a source of advice," Finance Research Letters, Elsevier, vol. 58(PA).
    2. Thérèse Lind & Ali Ahmed & Kenny Skagerlund & Camilla Strömbäck & Daniel Västfjäll & Gustav Tinghög, 2020. "Competence, Confidence, and Gender: The Role of Objective and Subjective Financial Knowledge in Household Finance," Journal of Family and Economic Issues, Springer, vol. 41(4), pages 626-638, December.
    3. Niszczota, Paweł & Błaszczyński, Jakub, 2024. "Hard to digest investments: People oppose investment in both conventional and cultured meat producers," Ecological Economics, Elsevier, vol. 218(C).
    4. Stone, Gregory & Walton, Stephanie & Zhang, Yibo (James), 2023. "The impact of online tax community advice on individual taxpayer decision making," Advances in accounting, Elsevier, vol. 63(C).
    5. Jiang Cheng & Lu Yu, 2019. "Life and health insurance consumption in China: demographic and environmental risks," The Geneva Papers on Risk and Insurance - Issues and Practice, Palgrave Macmillan;The Geneva Association, vol. 44(1), pages 67-101, January.
    6. Christelis, Dimitris & Dobrescu, Loretti I. & Motta, Alberto, 2020. "Early life conditions and financial risk-taking in older age," The Journal of the Economics of Ageing, Elsevier, vol. 17(C).
    7. Tullio Jappelli, 2010. "Economic Literacy: An International Comparison," Economic Journal, Royal Economic Society, vol. 120(548), pages 429-451, November.
    8. Lee, Boram & Rosenthal, Leonard & Veld, Chris & Veld-Merkoulova, Yulia, 2015. "Stock market expectations and risk aversion of individual investors," International Review of Financial Analysis, Elsevier, vol. 40(C), pages 122-131.
    9. Ni, Dan & Jiwen Song, Lynda & Zheng, Xiaoming & Zhu, Jinlong & Zhang, Mengyi & Xu, Lingxiao, 2022. "Extending a helping hand: How receiving gratitude makes a difference in employee performance during a crisis," Journal of Business Research, Elsevier, vol. 149(C), pages 967-982.
    10. Margaret Miller & Julia Reichelstein & Christian Salas & Bilal Zia, 2015. "Can You Help Someone Become Financially Capable? A Meta-Analysis of the Literature," The World Bank Research Observer, World Bank, vol. 30(2), pages 220-246.
    11. Anna Ispierto Maté, Irma Martínez García, Gloria Ruiz Suárez., 2021. "Educación financiera y decisiones de ahorro e inversión: un análisis de la Encuesta de Competencias Financieras (ECF)," CNMV Documentos de Trabajo CNMV Documentos de Trabaj, CNMV- Comisión Nacional del Mercado de Valores - Departamento de Estudios y Estadísticas.
    12. Julie Bayle-Cordier & Loïc Berger & Rayan Elatmani & Massimo Tavoni, 2023. "Breath, Love, Walk? The Impact of Mindfulness Interventions on Climate Policy Support and Environmental Attitudes," Sustainability, MDPI, vol. 15(13), pages 1-29, July.
    13. Alisdair McKay, 2011. "Household Saving Behavior and Social Security Privatization," Boston University - Department of Economics - Working Papers Series WP2011-027, Boston University - Department of Economics.
    14. Ranganathan, Kavitha & Lejarraga, Tomás, 2021. "Elicitation of risk preferences through satisficing," Journal of Behavioral and Experimental Finance, Elsevier, vol. 32(C).
    15. Goedde-Menke, Michael & Langer, Thomas & Pfingsten, Andreas, 2014. "Impact of the financial crisis on bank run risk – Danger of the days after," Journal of Banking & Finance, Elsevier, vol. 40(C), pages 522-533.
    16. Aman, Hiroyuki & Motonishi, Taizo & Ogawa, Kazuhito & Omori, Kozo, 2024. "The effect of financial literacy on long-term recognition and short-term trade in mutual funds: Evidence from Japan," International Review of Economics & Finance, Elsevier, vol. 89(PB), pages 762-783.
    17. Dr Samamba, Lennox Trivedi, PhD—Law, 2023. "Legal Aspects of Promoting the Participation of Collective Investment Schemes in Eastern and Southern African Frontier Securities Markets," International Journal of Research and Innovation in Social Science, International Journal of Research and Innovation in Social Science (IJRISS), vol. 7(4), pages 121-141, April.
    18. Lyn M. Van Swol & Paul Hangsan Ahn & Andrew Prahl & Zhenxing Gong, 2021. "Language Use in Group Discourse and Its Relationship to Group Processes," SAGE Open, , vol. 11(1), pages 21582440211, March.
    19. Bannier, Christina E. & Neubert, Milena, 2016. "Gender differences in financial risk taking: The role of financial literacy and risk tolerance," Economics Letters, Elsevier, vol. 145(C), pages 130-135.
    20. Dräger, Lena & Lamla, Michael J. & Pfajfar, Damjan, 2020. "The Hidden Heterogeneity of Inflation and Interest Rate Expectations: The Role of Preferences," Hannover Economic Papers (HEP) dp-666, Leibniz Universität Hannover, Wirtschaftswissenschaftliche Fakultät, revised Feb 2023.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2309.00649. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.