IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2411.01582.html
   My bibliography  Save this paper

Donald Trumps in the Virtual Polls: Simulating and Predicting Public Opinions in Surveys Using Large Language Models

Author

Listed:
  • Shapeng Jiang
  • Lijia Wei
  • Chen Zhang

Abstract

In recent years, large language models (LLMs) have attracted attention due to their ability to generate human-like text. As surveys and opinion polls remain key tools for gauging public attitudes, there is increasing interest in assessing whether LLMs can accurately replicate human responses. This study examines the potential of LLMs, specifically ChatGPT-4o, to replicate human responses in large-scale surveys and to predict election outcomes based on demographic data. Employing data from the World Values Survey (WVS) and the American National Election Studies (ANES), we assess the LLM's performance in two key tasks: simulating human responses and forecasting U.S. election results. In simulations, the LLM was tasked with generating synthetic responses for various socio-cultural and trust-related questions, demonstrating notable alignment with human response patterns across U.S.-China samples, though with some limitations on value-sensitive topics. In prediction tasks, the LLM was used to simulate voting behavior in past U.S. elections and predict the 2024 election outcome. Our findings show that the LLM replicates cultural differences effectively, exhibits in-sample predictive validity, and provides plausible out-of-sample forecasts, suggesting potential as a cost-effective supplement for survey-based research.

Suggested Citation

  • Shapeng Jiang & Lijia Wei & Chen Zhang, 2024. "Donald Trumps in the Virtual Polls: Simulating and Predicting Public Opinions in Surveys Using Large Language Models," Papers 2411.01582, arXiv.org.
  • Handle: RePEc:arx:papers:2411.01582
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2411.01582
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Erik Snowberg & Leeat Yariv, 2021. "Testing the Waters: Behavior across Participant Pools," American Economic Review, American Economic Association, vol. 111(2), pages 687-719, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Niculaescu, Corina E. & Sangiorgi, Ivan & Bell, Adrian R., 2023. "Does personal experience with COVID-19 impact investment decisions? Evidence from a survey of US retail investors," International Review of Financial Analysis, Elsevier, vol. 88(C).
    2. Gary Bolton & Kevin Breuer & Ben Greiner & Axel Ockenfels, 2023. "Fixing feedback revision rules in online markets," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 32(2), pages 247-256, April.
    3. Kesternich, Iris & Schumacher, Heiner & Siflinger, Bettina & Valder, Franziska, 2022. "Reservation wages and labor supply," Journal of Economic Behavior & Organization, Elsevier, vol. 194(C), pages 583-607.
    4. Dai, Zhixin & Zheng, Jiwei & Zizzo, Daniel John, 2024. "Theories of reasoning and focal point play with a matched non-student sample," China Economic Review, Elsevier, vol. 83(C).
    5. Matthew O. Jackson & Stephen M. Nei & Erik Snowberg & Leeat Yariv, 2022. "The Dynamics of Networks and Homophily," NBER Working Papers 30815, National Bureau of Economic Research, Inc.
    6. Abel Brodeur, Nikolai M. Cook, Anthony Heyes, 2022. "We Need to Talk about Mechanical Turk: What 22,989 Hypothesis Tests Tell Us about Publication Bias and p-Hacking in Online Experiments," LCERPA Working Papers am0133, Laurier Centre for Economic Research and Policy Analysis.
    7. Detemple, Julian & Kosfeld, Michael, 2023. "Fairness and Inequality in Institution Formation," IZA Discussion Papers 16464, Institute of Labor Economics (IZA).
    8. Brice Corgnet & Mark DeSantis & Christoph Siemroth, 2023. "Algorithmic Trading, Price Efficiency and Welfare: An Experimental Approach," Working Papers 2313, Groupe d'Analyse et de Théorie Economique Lyon St-Étienne (GATE Lyon St-Étienne), Université de Lyon.
    9. Ronayne, David & Sgroi, Daniel & Tuckwell, Anthony, 2021. "Evaluating the sunk cost effect," Journal of Economic Behavior & Organization, Elsevier, vol. 186(C), pages 318-327.
    10. Heger, Stephanie A. & Slonim, Robert, 2022. "Giving begets giving: Positive path dependence as moral consistency," Journal of Economic Behavior & Organization, Elsevier, vol. 204(C), pages 699-718.
    11. Marie Ferré & Stefanie Engel & Elisabeth Gsottbauer, 2023. "External validity of economic experiments on Agri‐environmental scheme design," Journal of Agricultural Economics, Wiley Blackwell, vol. 74(3), pages 661-685, September.
    12. Cook, Nikolai & Heyes, Anthony, 2022. "Pollution pictures: Psychological exposure to pollution impacts worker productivity in a large-scale field experiment," Journal of Environmental Economics and Management, Elsevier, vol. 114(C).
    13. Ernst Fehr & Gary Charness, 2023. "Social preferences: fundamental characteristics and economic consequences," ECON - Working Papers 432, Department of Economics - University of Zurich, revised Mar 2024.
    14. Thomas Epper & Julien Senn & Ernst Fehr, 2023. "The Missing Type: Where Are the Inequality Averse (Students)?," Working Papers 2023-iRisk-06, IESEG School of Management.
    15. Gächter, Simon & Lee, Kyeongtae & Sefton, Martin & Weber, Till O., 2024. "The role of payoff parameters for cooperation in the one-shot Prisoner's Dilemma," European Economic Review, Elsevier, vol. 166(C).
    16. Elias Bouacida & Renaud Foucart, 2022. "Rituals of Reason," Working Papers 344119591, Lancaster University Management School, Economics Department.
    17. Dariel, Aurelie & Riedl, Arno & Siegenthaler, Simon, 2021. "Referral hiring and wage formation in a market with adverse selection," Games and Economic Behavior, Elsevier, vol. 130(C), pages 109-130.
    18. Thomas Epper & Julien Senn & Ernst Fehr, 2023. "Social preferences across subject pools: students vs. general population," ECON - Working Papers 435, Department of Economics - University of Zurich, revised Jan 2024.
    19. Huynh, Luu Duc Toan & Stratmann, Philipp & Rilke, Rainer Michael, 2024. "No influence of simple moral awareness cues on cheating behaviour in an online experiment," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 108(C).
    20. Chapkovski, Philipp, 2022. "Interactive experiments in Toloka," MPRA Paper 111980, University Library of Munich, Germany.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2411.01582. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.