IDEAS home Printed from https://ideas.repec.org/p/cpr/ceprdp/13830.html
   My bibliography  Save this paper

Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya

Author

Listed:
  • Bauer, Michal
  • Chytilová, Julie
  • Miguel, Edward

Abstract

Can a short survey instrument reliably measure a range of fundamental economic preferences across diverse settings? We focus on survey questions that systematically predict behavior in incentivized experimental tasks among German university students (Becker et al. 2016) and were implemented among representative samples across the globe (Falk et al. 2018). This paper presents results of an experimental validation conducted among low-income individuals in Nairobi, Kenya. We find that quantitative survey measures -- hypothetical versions of experimental tasks -- of time preference, attitude to risk and altruism are good predictors of choices in incentivized experiments, suggesting these measures are broadly experimentally valid. At the same time, we find that qualitative questions -- self-assessments -- do not correlate with the experimental measures of preferences in the Kenyan sample. Thus, caution is needed before treating self-assessments as proxies of preferences in new contexts.

Suggested Citation

  • Bauer, Michal & Chytilová, Julie & Miguel, Edward, 2019. "Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya," CEPR Discussion Papers 13830, C.E.P.R. Discussion Papers.
    • Handle: RePEc:cpr:ceprdp:13830
    as

    Download full text from publisher

    File URL: https://cepr.org/publications/DP13830
    Download Restriction: CEPR Discussion Papers are free to download for our researchers, subscribers and members. If you fall into one of these categories but have trouble downloading our papers, please contact us at subscribers@cepr.org
    ---><---

    As the access to this document is restricted, you may want to look for a different version below or search for a different version of it.

    Other versions of this item:

    References listed on IDEAS

    as
    1. Armin Falk & Ernst Fehr & Urs Fischbacher, 2005. "Driving Forces Behind Informal Sanctions," Econometrica, Econometric Society, vol. 73(6), pages 2017-2030, November.
    2. Barasinska, Nataliya & Schäfer, Dorothea & Stephan, Andreas, 2012. "Individual Risk Attitudes and the Composition of Financial Portfolios: Evidence from German Household Portfolios," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 52(1), pages 1-14.
    3. Matthias Sutter & Martin G. Kocher & Daniela Glätzle-Rützler & Stefan T. Trautmann, 2013. "Impatience and Uncertainty: Experimental Decisions Predict Adolescents' Field Behavior," American Economic Review, American Economic Association, vol. 103(1), pages 510-531, February.
    4. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    5. Armin Falk & Ernst Fehr & Urs Fischbacher, "undated". "Driving Forces of Informal Sanctions," IEW - Working Papers 059, Institute for Empirical Research in Economics - University of Zurich.
    6. Bauer, Michal & Cahlíková, Jana & Celik Katreniak, Dagmara & Chytilová, Julie & Cingl, Lubomir & Želinský, Tomáš, 2018. "Anti-Social Behavior in Groups," IZA Discussion Papers 11944, Institute of Labor Economics (IZA).
    7. Falk, Armin & Hermle, Johannes, 2018. "Relationship of Gender Differences in Preferences to Economic Development and Gender Equality," IZA Discussion Papers 12059, Institute of Labor Economics (IZA).
    8. Stephan Meier & Charles Sprenger, 2010. "Present-Biased Preferences and Credit Card Borrowing," American Economic Journal: Applied Economics, American Economic Association, vol. 2(1), pages 193-210, January.
    9. Anke Becker & Benjamin Enke & Armin Falk, 2020. "Ancient Origins of the Global Variation in Economic Preferences," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 319-323, May.
    10. Sarah Baird & Joan Hamory Hicks & Michael Kremer & Edward Miguel, 2016. "Worms at Work: Long-run Impacts of a Child Health Investment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1637-1680.
    11. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    12. Bauernschuster, Stefan & Falck, Oliver & Heblich, Stephan & Suedekum, Jens & Lameli, Alfred, 2014. "Why are educated and risk-loving persons more mobile across regions?," Journal of Economic Behavior & Organization, Elsevier, vol. 98(C), pages 56-69.
    13. Abbink, Klaus & Sadrieh, Abdolkarim, 2009. "The pleasure of being nasty," Economics Letters, Elsevier, vol. 105(3), pages 306-308, December.
    14. Prediger, Sebastian & Vollan, Björn & Herrmann, Benedikt, 2014. "Resource scarcity and antisocial behavior," Journal of Public Economics, Elsevier, vol. 119(C), pages 1-9.
    15. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David B. Huffman & Uwe Sunde, 2017. "Global Evidence on Economic Preferences," NBER Working Papers 23943, National Bureau of Economic Research, Inc.
    16. Ernst Fehr & Karla Hoff & Mayuresh Kshetramade, 2008. "Spite and Development," American Economic Review, American Economic Association, vol. 98(2), pages 494-499, May.
    17. Christopher Blattman & Edward Miguel, 2010. "Civil War," Journal of Economic Literature, American Economic Association, vol. 48(1), pages 3-57, March.
    18. Rachid Laajaj & Karen Macours & Daniel Alejandro Pinzon Hernandez & Omar Arias & Samuel D. Gosling & Jeff Potter & Marta Rubio-Codina & Renos Vakis, 2019. "Challenges to capture the big five personality traits in non-WEIRD populations," PSE-Ecole d'économie de Paris (Postprint) halshs-02306945, HAL.
    19. Xiaohao Ding & Joop Hartog & Yuze Sun, 2010. "Can we measure Individual Risk Attitudes in a Survey?," Tinbergen Institute Discussion Papers 10-027/3, Tinbergen Institute.
    20. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    21. Fouarge, Didier & Kriechel, Ben & Dohmen, Thomas, 2014. "Occupational sorting of school graduates: The role of economic preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 106(C), pages 335-351.
    22. Bonin, Holger & Dohmen, Thomas & Falk, Armin & Huffman, David & Sunde, Uwe, 2007. "Cross-sectional earnings risk and occupational sorting: The role of risk attitudes," Labour Economics, Elsevier, vol. 14(6), pages 926-937, December.
    23. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    24. repec:nas:journl:v:115:y:2018:p:4881-4886 is not listed on IDEAS
    25. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    26. Ding, Xiaohao & Hartog, Joop & Sun, Yuze, 2010. "Can We Measure Individual Risk Attitudes in a Survey?," IZA Discussion Papers 4807, Institute of Labor Economics (IZA).
    27. Joseph Henrich & Steve J. Heine & Ara Norenzayan, 2010. "The Weirdest People in the World?," RatSWD Working Papers 139, German Data Forum (RatSWD).
    28. David A. Jaeger & Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Holger Bonin, 2010. "Direct Evidence on Risk Attitudes and Migration," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 684-689, August.
    29. Bernd Hardeweg & Lukas Menkhoff & Hermann Waibel, 2013. "Experimentally Validated Survey Evidence on Individual Risk Attitudes in Rural Thailand," Economic Development and Cultural Change, University of Chicago Press, vol. 61(4), pages 859-888.
    30. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    31. Ernst Fehr & Urs Fischbacher, 2003. "The nature of human altruism," Nature, Nature, vol. 425(6960), pages 785-791, October.
    32. Ferdinand M. Vieider & Mathieu Lefebvre & Ranoua Bouchouicha & Thorsten Chmura & Rustamdjan Hakimov & Michal Krawczyk & Peter Martinsson, 2015. "Common Components Of Risk And Uncertainty Attitudes Across Contexts And Domains: Evidence From 30 Countries," Journal of the European Economic Association, European Economic Association, vol. 13(3), pages 421-452, June.
    33. Zacharias Maniadis & Fabio Tufano & John A. List, 2014. "One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects," American Economic Review, American Economic Association, vol. 104(1), pages 277-290, January.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Michal Bauer & Jana Cahlíková & Julie Chytilová & Gérard Roland & Tomáš Želinský, 2023. "Shifting Punishment onto Minorities: Experimental Evidence of Scapegoating," The Economic Journal, Royal Economic Society, vol. 133(652), pages 1626-1640.
    2. Leites, Martín & Rivero, Analía & Salas, Gonzalo, 2024. "The positionality of goods and the positional concern’s origin," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 109(C).
    3. Uwe Sunde & Thomas Dohmen & Benjamin Enke & Armin Falkbriq & David Huffman & Gerrit Meyerheim, 2022. "Patience and Comparative Development," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(5), pages 2806-2840.
    4. Alexandros Karakostas & Nhu Tran & Daniel John Zizzo, 2022. "Experimental Insights on Anti-Social Behavior: Two Meta-Analyses," Discussion Papers Series 658, School of Economics, University of Queensland, Australia.
    5. Daniel Engler & Gunnar Gutsche & Amantia Simixhiu & Andreas Ziegler, 2022. "Social norms and individual climate protection activities: A framed field experiment for Germany," MAGKS Papers on Economics 202230, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    6. Bérgolo, Marcelo & Burdin, Gabriel & Burone, Santiago & De Rosa, Mauricio & Giaccobasso, Matias & Leites, Martin, 2022. "Dissecting inequality-averse preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 782-802.
    7. Marcelo Bérgolo & Gabriel Burdín & Santiago Burone & Mauricio de Rosa & Matías Giaccobasso & Martín Leites, 2020. "Dissecting Inequality-Averse Preferences," Documentos de Trabajo (working papers) 20-19, Instituto de Economía - IECON.
    8. Lubomir Cingl & Tomas Lichard & Tomas Miklanek, 2022. "Mist Over a Meadow: Tax Designation Effects on Compliance," CERGE-EI Working Papers wp725, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    9. Beraldo, Sergio & Piacenza, Massimiliano & Turati, Gilberto, 2022. "The importance of the future when deciding levels of personal responsibility and demand for redistribution," Economic Modelling, Elsevier, vol. 116(C).
    10. Pierre-Guillaume Méon & Robin Rampaer & David Raymaekers, 2021. "One-minute earthquake, years of patience: Evidence from Mexico on the effect of earthquake exposure on time preference," Working Papers CEB 21-015, ULB -- Universite Libre de Bruxelles.
    11. Vojtěch Bartoš & Ian Levely & Vojtech Bartos, 2023. "Measuring Social Preferences in Developing Economies," CESifo Working Paper Series 10744, CESifo.
    12. Bietenbeck, Jan & Sunde, Uwe & Thiemann, Petra, 2023. "Recession Experiences during Early Adulthood Shape Prosocial Attitudes Later in Life," IZA Discussion Papers 16490, Institute of Labor Economics (IZA).
    13. Daniel L. Chen & Lubomir Cingl & Arnaud Philippe & Michal Soltes, 2024. "Exploring Inmates’ Perceptions, Attitudes, and Behavior: Implications for Theories of Crime," CERGE-EI Working Papers wp779, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    14. Engler, Daniel & Ziegler, Andreas & Gutsche, Gunnar & Simixhiu, Amantia, 2023. "Social Norms and Individual Climate Protection Activities: A Framed Field Experiment for Germany," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277662, Verein für Socialpolitik / German Economic Association.
    15. Haverkamp, Thilo K.G. & Welsch, Heinz & Ziegler, Andreas, 2023. "The relationship between climate protection activities, economic preferences, and life satisfaction: Empirical evidence for Germany," Energy Economics, Elsevier, vol. 128(C).
    16. Sanjaya, Muhammad Ryan, 2023. "Antisocial behavior in experiments: What have we learned from the past two decades?," Research in Economics, Elsevier, vol. 77(1), pages 104-115.
    17. Cingl, Lubomír & Lichard, Tomáš & Miklánek, Tomáš, 2023. "Tax designation effects on compliance: An online experiment with taxpayers," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 615-633.
    18. Kosfeld, Michael & Sharafi, Zahra, 2022. "The Preference Survey Module: New Evidence on Social Preferences from Tehran," IZA Discussion Papers 15006, Institute of Labor Economics (IZA).
    19. Benjamin Enke & Ricardo Rodríguez-Padilla & Florian Zimmermann, 2022. "Moral Universalism: Measurement and Economic Relevance," Management Science, INFORMS, vol. 68(5), pages 3590-3603, May.
    20. Tomáš Želinský, 2021. "Intertemporal Choices of Children and Adults from Poor Roma Communities: A Case Study from Slovakia," Eastern European Economics, Taylor & Francis Journals, vol. 59(4), pages 378-405, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    2. Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
    3. Bühler, Dorothee & Sharma, Rasadhika & Stein, Wiebke, 2020. "Occupational Attainment and Earnings in Southeast Asia: The Role of Non-cognitive Skills," Labour Economics, Elsevier, vol. 67(C).
    4. Müller, Stephan & Rau, Holger A., 2020. "Economic preferences and compliance in the social stress test of the Corona crisis," University of Göttingen Working Papers in Economics 391, University of Goettingen, Department of Economics.
    5. Joshua Tasoff & Wenjie Zhang, 2022. "The Performance of Time-Preference and Risk-Preference Measures in Surveys," Management Science, INFORMS, vol. 68(2), pages 1149-1173, February.
    6. Müller, Stephan & Rau, Holger A., 2021. "Economic preferences and compliance in the social stress test of the COVID-19 crisis," Journal of Public Economics, Elsevier, vol. 194(C).
    7. Breitkopf, Laura & Chowdhury, Shyamal K. & Priyam, Shambhavi & Schildberg-Hörisch, Hannah & Sutter, Matthias, 2020. "Do economic preferences of children predict behavior?," DICE Discussion Papers 342, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    8. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    9. Andrej Gill & Florian Hett & Johannes Tischer, 2022. "Time Inconsistency and Overdraft Use: Evidence from Transaction Data and Behavioral Measurement Experiments," Working Papers 2205, Gutenberg School of Management and Economics, Johannes Gutenberg-Universität Mainz.
    10. Sepahvand, Mohammad & Shahbazian, Roujman, 2017. "Individual’s Risk Attitudes in sub-Saharan Africa: Determinants and Reliability of Self-reported Risk in Burkina Faso," Working Paper Series 2017:11, Uppsala University, Department of Economics.
    11. Dohmen, Thomas & Quercia, Simone & Willrodt, Jana, 2018. "Willingness to Take Risk: The Role of Risk Conception and Optimism," IZA Discussion Papers 11642, Institute of Labor Economics (IZA).
    12. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    13. Kawamura, Tetsuya & Mori, Tomoharu & Motonishi, Taizo & Ogawa, Kazuhito, 2021. "Is Financial Literacy Dangerous? Financial Literacy, Behavioral Factors, and Financial Choices of Households," Journal of the Japanese and International Economies, Elsevier, vol. 60(C).
    14. Thomas Dohmen & Simone Quercia & Jana Willrodt, 2023. "On the psychology of the relation between optimism and risk taking," Journal of Risk and Uncertainty, Springer, vol. 67(2), pages 193-214, October.
    15. Dániel Horn & Hubert János Kiss, 2020. "Time preferences and their life outcome correlates: Evidence from a representative survey," PLOS ONE, Public Library of Science, vol. 15(7), pages 1-26, July.
    16. Tim Friehe & Markus Pannenberg, 2020. "Time preferences and political regimes: evidence from reunified Germany," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(1), pages 349-387, January.
    17. Dohmen, Thomas & Lehmann, Hartmut & Pignatti, Norberto, 2016. "Time-varying individual risk attitudes over the Great Recession: A comparison of Germany and Ukraine," Journal of Comparative Economics, Elsevier, vol. 44(1), pages 182-200.
    18. Gill, Andrej & Hett, Florian & Tischer, Johannes, 2022. "Time inconsistency and overdraft use: Evidence from transaction data and behavioral measurement experiments," SAFE Working Paper Series 347, Leibniz Institute for Financial Research SAFE.
    19. Carpenter, Jeffrey, 2021. "The shape of warm glow: Field experimental evidence from a fundraiser," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 555-574.
    20. Elisa Cavatorta & David Schröder, 2019. "Measuring ambiguity preferences: A new ambiguity preference survey module," Journal of Risk and Uncertainty, Springer, vol. 58(1), pages 71-100, February.

    More about this item

    Keywords

    Preference measurement; Experiment; Survey; Validation;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D90 - Microeconomics - - Micro-Based Behavioral Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:cpr:ceprdp:13830. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: the person in charge (email available below). General contact details of provider: https://www.cepr.org .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.