IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp12450.html
   My bibliography  Save this paper

Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya

Author

Listed:
  • Bauer, Michal

    (Charles University, Prague)

  • Chytilová, Julie

    (Charles University, Prague)

  • Miguel, Edward

    (University of California, Berkeley)

Abstract

Can a short survey instrument reliably measure a range of fundamental economic preferences across diverse settings? We focus on survey questions that systematically predict behavior in incentivized experimental tasks among German university students (Becker et al. 2016) and were implemented among representative samples across the globe (Falk et al. 2018). This paper presents results of an experimental validation conducted among low-income individuals in Nairobi, Kenya. We find that quantitative survey measures - hypothetical versions of experimental tasks - of time preference, attitude to risk and altruism are good predictors of choices in incentivized experiments, suggesting these measures are broadly experimentally valid. At the same time, we find that qualitative questions - self-assessments - do not correlate with the experimental measures of preferences in the Kenyan sample. Thus, caution is needed before treating self-assessments as proxies of preferences in new contexts.

Suggested Citation

  • Bauer, Michal & Chytilová, Julie & Miguel, Edward, 2019. "Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya," IZA Discussion Papers 12450, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp12450
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp12450.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Armin Falk & Ernst Fehr & Urs Fischbacher, 2005. "Driving Forces Behind Informal Sanctions," Econometrica, Econometric Society, vol. 73(6), pages 2017-2030, November.
    2. Barasinska, Nataliya & Schäfer, Dorothea & Stephan, Andreas, 2012. "Individual Risk Attitudes and the Composition of Financial Portfolios: Evidence from German Household Portfolios," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 52(1), pages 1-14.
    3. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    4. Armin Falk & Ernst Fehr & Urs Fischbacher, "undated". "Driving Forces of Informal Sanctions," IEW - Working Papers 059, Institute for Empirical Research in Economics - University of Zurich.
    5. Anke Becker & Benjamin Enke & Armin Falk, 2020. "Ancient Origins of the Global Variation in Economic Preferences," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 319-323, May.
    6. Matthias Sutter & Martin G. Kocher & Daniela Glätzle-Rützler & Stefan T. Trautmann, 2013. "Impatience and Uncertainty: Experimental Decisions Predict Adolescents' Field Behavior," American Economic Review, American Economic Association, vol. 103(1), pages 510-531, February.
    7. Bauernschuster, Stefan & Falck, Oliver & Heblich, Stephan & Suedekum, Jens & Lameli, Alfred, 2014. "Why are educated and risk-loving persons more mobile across regions?," Journal of Economic Behavior & Organization, Elsevier, vol. 98(C), pages 56-69.
    8. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    9. Abbink, Klaus & Sadrieh, Abdolkarim, 2009. "The pleasure of being nasty," Economics Letters, Elsevier, vol. 105(3), pages 306-308, December.
    10. Stephan Meier & Charles Sprenger, 2010. "Present-Biased Preferences and Credit Card Borrowing," American Economic Journal: Applied Economics, American Economic Association, vol. 2(1), pages 193-210, January.
    11. Ernst Fehr & Karla Hoff & Mayuresh Kshetramade, 2008. "Spite and Development," American Economic Review, American Economic Association, vol. 98(2), pages 494-499, May.
    12. Xiaohao Ding & Joop Hartog & Yuze Sun, 2010. "Can we measure Individual Risk Attitudes in a Survey?," Tinbergen Institute Discussion Papers 10-027/3, Tinbergen Institute.
    13. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    14. Fouarge, Didier & Kriechel, Ben & Dohmen, Thomas, 2014. "Occupational sorting of school graduates: The role of economic preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 106(C), pages 335-351.
    15. Bonin, Holger & Dohmen, Thomas & Falk, Armin & Huffman, David & Sunde, Uwe, 2007. "Cross-sectional earnings risk and occupational sorting: The role of risk attitudes," Labour Economics, Elsevier, vol. 14(6), pages 926-937, December.
    16. Rachid Laajaj & Karen Macours & Daniel Alejandro Pinzon Hernandez & Omar Arias & Samuel D. Gosling & Jeff Potter & Marta Rubio-Codina & Renos Vakis, 2019. "Challenges to capture the big five personality traits in non-WEIRD populations," Post-Print halshs-02306945, HAL.
    17. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    18. repec:nas:journl:v:115:y:2018:p:4881-4886 is not listed on IDEAS
    19. Sarah Baird & Joan Hamory Hicks & Michael Kremer & Edward Miguel, 2016. "Worms at Work: Long-run Impacts of a Child Health Investment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1637-1680.
    20. Joseph Henrich & Steve J. Heine & Ara Norenzayan, 2010. "The Weirdest People in the World?," RatSWD Working Papers 139, German Data Forum (RatSWD).
    21. David A. Jaeger & Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Holger Bonin, 2010. "Direct Evidence on Risk Attitudes and Migration," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 684-689, August.
    22. Bernd Hardeweg & Lukas Menkhoff & Hermann Waibel, 2013. "Experimentally Validated Survey Evidence on Individual Risk Attitudes in Rural Thailand," Economic Development and Cultural Change, University of Chicago Press, vol. 61(4), pages 859-888.
    23. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    24. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David B. Huffman & Uwe Sunde, 2017. "Global Evidence on Economic Preferences," NBER Working Papers 23943, National Bureau of Economic Research, Inc.
    25. Ernst Fehr & Urs Fischbacher, 2003. "The nature of human altruism," Nature, Nature, vol. 425(6960), pages 785-791, October.
    26. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    27. Zacharias Maniadis & Fabio Tufano & John A. List, 2014. "One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects," American Economic Review, American Economic Association, vol. 104(1), pages 277-290, January.
    28. Bauer, Michal & Cahlíková, Jana & Celik Katreniak, Dagmara & Chytilová, Julie & Cingl, Lubomir & Želinský, Tomáš, 2018. "Anti-Social Behavior in Groups," IZA Discussion Papers 11944, Institute of Labor Economics (IZA).
    29. Falk, Armin & Hermle, Johannes, 2018. "Relationship of Gender Differences in Preferences to Economic Development and Gender Equality," IZA Discussion Papers 12059, Institute of Labor Economics (IZA).
    30. Prediger, Sebastian & Vollan, Björn & Herrmann, Benedikt, 2014. "Resource scarcity and antisocial behavior," Journal of Public Economics, Elsevier, vol. 119(C), pages 1-9.
    31. Christopher Blattman & Edward Miguel, 2010. "Civil War," Journal of Economic Literature, American Economic Association, vol. 48(1), pages 3-57, March.
    32. Ding, Xiaohao & Hartog, Joop & Sun, Yuze, 2010. "Can We Measure Individual Risk Attitudes in a Survey?," IZA Discussion Papers 4807, Institute of Labor Economics (IZA).
    33. Ferdinand M. Vieider & Mathieu Lefebvre & Ranoua Bouchouicha & Thorsten Chmura & Rustamdjan Hakimov & Michal Krawczyk & Peter Martinsson, 2015. "Common Components Of Risk And Uncertainty Attitudes Across Contexts And Domains: Evidence From 30 Countries," Journal of the European Economic Association, European Economic Association, vol. 13(3), pages 421-452, June.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Leites, Martín & Rivero, Analía & Salas, Gonzalo, 2024. "The positionality of goods and the positional concern’s origin," Journal of Behavioral and Experimental Economics (formerly The Journal of Socio-Economics), Elsevier, vol. 109(C).
    2. Uwe Sunde & Thomas Dohmen & Benjamin Enke & Armin Falkbriq & David Huffman & Gerrit Meyerheim, 2022. "Patience and Comparative Development," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 89(5), pages 2806-2840.
    3. Daniel Engler & Gunnar Gutsche & Amantia Simixhiu & Andreas Ziegler, 2022. "Social norms and individual climate protection activities: A framed field experiment for Germany," MAGKS Papers on Economics 202230, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    4. Marcelo Bérgolo & Gabriel Burdín & Santiago Burone & Mauricio de Rosa & Matías Giaccobasso & Martín Leites, 2020. "Dissecting Inequality-Averse Preferences," Documentos de Trabajo (working papers) 20-19, Instituto de Economía - IECON.
    5. Lubomir Cingl & Tomas Lichard & Tomas Miklanek, 2022. "Mist Over a Meadow: Tax Designation Effects on Compliance," CERGE-EI Working Papers wp725, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    6. Beraldo, Sergio & Piacenza, Massimiliano & Turati, Gilberto, 2022. "The importance of the future when deciding levels of personal responsibility and demand for redistribution," Economic Modelling, Elsevier, vol. 116(C).
    7. Michal Bauer & Jana Cahlíková & Julie Chytilová & Gérard Roland & Tomáš Želinský, 2023. "Shifting Punishment onto Minorities: Experimental Evidence of Scapegoating," The Economic Journal, Royal Economic Society, vol. 133(652), pages 1626-1640.
    8. Bietenbeck, Jan & Sunde, Uwe & Thiemann, Petra, 2023. "Recession Experiences during Early Adulthood Shape Prosocial Attitudes Later in Life," IZA Discussion Papers 16490, Institute of Labor Economics (IZA).
    9. Daniel L. Chen & Lubomir Cingl & Arnaud Philippe & Michal Soltes, 2024. "Exploring Inmates’ Perceptions, Attitudes, and Behavior: Implications for Theories of Crime," CERGE-EI Working Papers wp779, The Center for Economic Research and Graduate Education - Economics Institute, Prague.
    10. Sanjaya, Muhammad Ryan, 2023. "Antisocial behavior in experiments: What have we learned from the past two decades?," Research in Economics, Elsevier, vol. 77(1), pages 104-115.
    11. Cingl, Lubomír & Lichard, Tomáš & Miklánek, Tomáš, 2023. "Tax designation effects on compliance: An online experiment with taxpayers," Journal of Economic Behavior & Organization, Elsevier, vol. 214(C), pages 615-633.
    12. Benjamin Enke & Ricardo Rodríguez-Padilla & Florian Zimmermann, 2022. "Moral Universalism: Measurement and Economic Relevance," Management Science, INFORMS, vol. 68(5), pages 3590-3603, May.
    13. Alexandros Karakostas & Nhu Tran & Daniel John Zizzo, 2022. "Experimental Insights on Anti-Social Behavior: Two Meta-Analyses," Discussion Papers Series 658, School of Economics, University of Queensland, Australia.
    14. Bérgolo, Marcelo & Burdin, Gabriel & Burone, Santiago & De Rosa, Mauricio & Giaccobasso, Matias & Leites, Martin, 2022. "Dissecting inequality-averse preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 782-802.
    15. Pierre-Guillaume Méon & Robin Rampaer & David Raymaekers, 2021. "One-minute earthquake, years of patience: Evidence from Mexico on the effect of earthquake exposure on time preference," Working Papers CEB 21-015, ULB -- Universite Libre de Bruxelles.
    16. Vojtěch Bartoš & Ian Levely & Vojtech Bartos, 2023. "Measuring Social Preferences in Developing Economies," CESifo Working Paper Series 10744, CESifo.
    17. Engler, Daniel & Ziegler, Andreas & Gutsche, Gunnar & Simixhiu, Amantia, 2023. "Social Norms and Individual Climate Protection Activities: A Framed Field Experiment for Germany," VfS Annual Conference 2023 (Regensburg): Growth and the "sociale Frage" 277662, Verein für Socialpolitik / German Economic Association.
    18. Haverkamp, Thilo K.G. & Welsch, Heinz & Ziegler, Andreas, 2023. "The relationship between climate protection activities, economic preferences, and life satisfaction: Empirical evidence for Germany," Energy Economics, Elsevier, vol. 128(C).
    19. Kosfeld, Michael & Sharafi, Zahra, 2022. "The Preference Survey Module: New Evidence on Social Preferences from Tehran," IZA Discussion Papers 15006, Institute of Labor Economics (IZA).
    20. Tomáš Želinský, 2021. "Intertemporal Choices of Children and Adults from Poor Roma Communities: A Case Study from Slovakia," Eastern European Economics, Taylor & Francis Journals, vol. 59(4), pages 378-405, July.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    2. Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
    3. Bühler, Dorothee & Sharma, Rasadhika & Stein, Wiebke, 2020. "Occupational Attainment and Earnings in Southeast Asia: The Role of Non-cognitive Skills," Labour Economics, Elsevier, vol. 67(C).
    4. Joshua Tasoff & Wenjie Zhang, 2022. "The Performance of Time-Preference and Risk-Preference Measures in Surveys," Management Science, INFORMS, vol. 68(2), pages 1149-1173, February.
    5. Müller, Stephan & Rau, Holger A., 2021. "Economic preferences and compliance in the social stress test of the COVID-19 crisis," Journal of Public Economics, Elsevier, vol. 194(C).
    6. Müller, Stephan & Rau, Holger A., 2020. "Economic preferences and compliance in the social stress test of the Corona crisis," University of Göttingen Working Papers in Economics 391, University of Goettingen, Department of Economics.
    7. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    8. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    9. Thomas Dohmen & Simone Quercia & Jana Willrodt, 2018. "Willingness to take risk: The role of risk conception and optimism," CRC TR 224 Discussion Paper Series crctr224_2018_023, University of Bonn and University of Mannheim, Germany.
    10. Dohmen, Thomas & Lehmann, Hartmut & Pignatti, Norberto, 2016. "Time-varying individual risk attitudes over the Great Recession: A comparison of Germany and Ukraine," Journal of Comparative Economics, Elsevier, vol. 44(1), pages 182-200.
    11. Gill, Andrej & Hett, Florian & Tischer, Johannes, 2022. "Time inconsistency and overdraft use: Evidence from transaction data and behavioral measurement experiments," Discussion Papers 18/2022, Deutsche Bundesbank.
    12. Carole Treibich, 2015. "Are Survey Risk Aversion Measurements Adequate in a Low Income Context?," AMSE Working Papers 1517, Aix-Marseille School of Economics, France.
    13. Carpenter, Jeffrey, 2021. "The shape of warm glow: Field experimental evidence from a fundraiser," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 555-574.
    14. Tim Friehe & Markus Pannenberg, 2020. "Time preferences and political regimes: evidence from reunified Germany," Journal of Population Economics, Springer;European Society for Population Economics, vol. 33(1), pages 349-387, January.
    15. Menkhoff, Lukas & Sakha, Sahra, 2014. "Multiple-item risk measures," Kiel Working Papers 1980, Kiel Institute for the World Economy (IfW Kiel).
    16. van Huizen, Thomas & Alessie, Rob, 2019. "Risk aversion and job mobility," Journal of Economic Behavior & Organization, Elsevier, vol. 164(C), pages 91-106.
    17. Thomas Dohmen & Simone Quercia & Jana Willrodt, 2023. "On the psychology of the relation between optimism and risk taking," Journal of Risk and Uncertainty, Springer, vol. 67(2), pages 193-214, October.
    18. Mohammad H. Sepahvand & Roujman Shahbazian, 2021. "Sibling correlation in risk attitudes: evidence from Burkina Faso," The Journal of Economic Inequality, Springer;Society for the Study of Economic Inequality, vol. 19(1), pages 45-72, March.
    19. Andreas Ziegler, 2020. "New Ecological Paradigm meets behavioral economics: On the relationship between environmental values and economic preferences," MAGKS Papers on Economics 202020, Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung).
    20. Bašić, Zvonimir & Bortolotti, Stefania & Salicath, Daniel & Schmidt, Stefan & Schneider, Sebastian O. & Sutter, Matthias, 2024. "One Size Fits All? The Interplay of Incentives, Effort Provision, and Personality," IZA Discussion Papers 17287, Institute of Labor Economics (IZA).

    More about this item

    Keywords

    experiment; preference measurement; survey; validation;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D90 - Microeconomics - - Micro-Based Behavioral Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp12450. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.