IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp12450.html
   My bibliography  Save this paper

Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya

Author

Listed:
  • Bauer, Michal

    (Charles University, Prague)

  • Chytilová, Julie

    (Charles University, Prague)

  • Miguel, Edward

    (University of California, Berkeley)

Abstract

Can a short survey instrument reliably measure a range of fundamental economic preferences across diverse settings? We focus on survey questions that systematically predict behavior in incentivized experimental tasks among German university students (Becker et al. 2016) and were implemented among representative samples across the globe (Falk et al. 2018). This paper presents results of an experimental validation conducted among low-income individuals in Nairobi, Kenya. We find that quantitative survey measures - hypothetical versions of experimental tasks - of time preference, attitude to risk and altruism are good predictors of choices in incentivized experiments, suggesting these measures are broadly experimentally valid. At the same time, we find that qualitative questions - self-assessments - do not correlate with the experimental measures of preferences in the Kenyan sample. Thus, caution is needed before treating self-assessments as proxies of preferences in new contexts.

Suggested Citation

  • Bauer, Michal & Chytilová, Julie & Miguel, Edward, 2019. "Using Survey Questions to Measure Preferences: Lessons from an Experimental Validation in Kenya," IZA Discussion Papers 12450, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp12450
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp12450.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Armin Falk & Ernst Fehr & Urs Fischbacher, 2005. "Driving Forces Behind Informal Sanctions," Econometrica, Econometric Society, vol. 73(6), pages 2017-2030, November.
    2. Barasinska, Nataliya & Schäfer, Dorothea & Stephan, Andreas, 2012. "Individual Risk Attitudes and the Composition of Financial Portfolios: Evidence from German Household Portfolios," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 52(1), pages 1-14.
    3. Garret Christensen & Edward Miguel, 2018. "Transparency, Reproducibility, and the Credibility of Economics Research," Journal of Economic Literature, American Economic Association, vol. 56(3), pages 920-980, September.
    4. Armin Falk & Ernst Fehr & Urs Fischbacher, "undated". "Driving Forces of Informal Sanctions," IEW - Working Papers 059, Institute for Empirical Research in Economics - University of Zurich.
    5. Anke Becker & Benjamin Enke & Armin Falk, 2020. "Ancient Origins of the Global Variation in Economic Preferences," AEA Papers and Proceedings, American Economic Association, vol. 110, pages 319-323, May.
    6. Matthias Sutter & Martin G. Kocher & Daniela Glätzle-Rützler & Stefan T. Trautmann, 2013. "Impatience and Uncertainty: Experimental Decisions Predict Adolescents' Field Behavior," American Economic Review, American Economic Association, vol. 103(1), pages 510-531, February.
    7. Bauernschuster, Stefan & Falck, Oliver & Heblich, Stephan & Suedekum, Jens & Lameli, Alfred, 2014. "Why are educated and risk-loving persons more mobile across regions?," Journal of Economic Behavior & Organization, Elsevier, vol. 98(C), pages 56-69.
    8. Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Jürgen Schupp & Gert G. Wagner, 2011. "Individual Risk Attitudes: Measurement, Determinants, And Behavioral Consequences," Journal of the European Economic Association, European Economic Association, vol. 9(3), pages 522-550, June.
    9. Abbink, Klaus & Sadrieh, Abdolkarim, 2009. "The pleasure of being nasty," Economics Letters, Elsevier, vol. 105(3), pages 306-308, December.
    10. Stephan Meier & Charles Sprenger, 2010. "Present-Biased Preferences and Credit Card Borrowing," American Economic Journal: Applied Economics, American Economic Association, vol. 2(1), pages 193-210, January.
    11. Ernst Fehr & Karla Hoff & Mayuresh Kshetramade, 2008. "Spite and Development," American Economic Review, American Economic Association, vol. 98(2), pages 494-499, May.
    12. Xiaohao Ding & Joop Hartog & Yuze Sun, 2010. "Can we measure Individual Risk Attitudes in a Survey?," Tinbergen Institute Discussion Papers 10-027/3, Tinbergen Institute.
    13. Nava Ashraf & Dean Karlan & Wesley Yin, 2006. "Tying Odysseus to the Mast: Evidence From a Commitment Savings Product in the Philippines," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 121(2), pages 635-672.
    14. Fouarge, Didier & Kriechel, Ben & Dohmen, Thomas, 2014. "Occupational sorting of school graduates: The role of economic preferences," Journal of Economic Behavior & Organization, Elsevier, vol. 106(C), pages 335-351.
    15. Bonin, Holger & Dohmen, Thomas & Falk, Armin & Huffman, David & Sunde, Uwe, 2007. "Cross-sectional earnings risk and occupational sorting: The role of risk attitudes," Labour Economics, Elsevier, vol. 14(6), pages 926-937, December.
    16. Rachid Laajaj & Karen Macours & Daniel Alejandro Pinzon Hernandez & Omar Arias & Samuel D. Gosling & Jeff Potter & Marta Rubio-Codina & Renos Vakis, 2019. "Challenges to capture the big five personality traits in non-WEIRD populations," Post-Print halshs-02306945, HAL.
    17. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    18. repec:nas:journl:v:115:y:2018:p:4881-4886 is not listed on IDEAS
    19. Sarah Baird & Joan Hamory Hicks & Michael Kremer & Edward Miguel, 2016. "Worms at Work: Long-run Impacts of a Child Health Investment," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 131(4), pages 1637-1680.
    20. Joseph Henrich & Steve J. Heine & Ara Norenzayan, 2010. "The Weirdest People in the World?," RatSWD Working Papers 139, German Data Forum (RatSWD).
    21. David A. Jaeger & Thomas Dohmen & Armin Falk & David Huffman & Uwe Sunde & Holger Bonin, 2010. "Direct Evidence on Risk Attitudes and Migration," The Review of Economics and Statistics, MIT Press, vol. 92(3), pages 684-689, August.
    22. Bernd Hardeweg & Lukas Menkhoff & Hermann Waibel, 2013. "Experimentally Validated Survey Evidence on Individual Risk Attitudes in Rural Thailand," Economic Development and Cultural Change, University of Chicago Press, vol. 61(4), pages 859-888.
    23. Edward Miguel & Michael Kremer, 2004. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities," Econometrica, Econometric Society, vol. 72(1), pages 159-217, January.
    24. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David B. Huffman & Uwe Sunde, 2017. "Global Evidence on Economic Preferences," NBER Working Papers 23943, National Bureau of Economic Research, Inc.
    25. Ernst Fehr & Urs Fischbacher, 2003. "The nature of human altruism," Nature, Nature, vol. 425(6960), pages 785-791, October.
    26. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    27. Zacharias Maniadis & Fabio Tufano & John A. List, 2014. "One Swallow Doesn't Make a Summer: New Evidence on Anchoring Effects," American Economic Review, American Economic Association, vol. 104(1), pages 277-290, January.
    28. Bauer, Michal & Cahlíková, Jana & Celik Katreniak, Dagmara & Chytilová, Julie & Cingl, Lubomir & Želinský, Tomáš, 2018. "Anti-Social Behavior in Groups," IZA Discussion Papers 11944, Institute of Labor Economics (IZA).
    29. Falk, Armin & Hermle, Johannes, 2018. "Relationship of Gender Differences in Preferences to Economic Development and Gender Equality," IZA Discussion Papers 12059, Institute of Labor Economics (IZA).
    30. Prediger, Sebastian & Vollan, Björn & Herrmann, Benedikt, 2014. "Resource scarcity and antisocial behavior," Journal of Public Economics, Elsevier, vol. 119(C), pages 1-9.
    31. Christopher Blattman & Edward Miguel, 2010. "Civil War," Journal of Economic Literature, American Economic Association, vol. 48(1), pages 3-57, March.
    32. Ding, Xiaohao & Hartog, Joop & Sun, Yuze, 2010. "Can We Measure Individual Risk Attitudes in a Survey?," IZA Discussion Papers 4807, Institute of Labor Economics (IZA).
    33. Ferdinand M. Vieider & Mathieu Lefebvre & Ranoua Bouchouicha & Thorsten Chmura & Rustamdjan Hakimov & Michal Krawczyk & Peter Martinsson, 2015. "Common Components Of Risk And Uncertainty Attitudes Across Contexts And Domains: Evidence From 30 Countries," Journal of the European Economic Association, European Economic Association, vol. 13(3), pages 421-452, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Armin Falk & Anke Becker & Thomas Dohmen & David Huffman & Uwe Sunde, 2023. "The Preference Survey Module: A Validated Instrument for Measuring Risk, Time, and Social Preferences," Management Science, INFORMS, vol. 69(4), pages 1935-1950, April.
    2. Michael Kosfeld & Zahra Sharafi & Maíra Sontag González & Na Zou, 2025. "Measuring Economic Preferences with Surveys and Behavioral Experiments," CESifo Working Paper Series 11631, CESifo.
    3. Kosfeld, Michael & Sharafi, Zahra & Sontag González, Maíra & Zou, Na, 2025. "Measuring Economic Preferences with Surveys and Behavioral Experiments," IZA Discussion Papers 17608, Institute of Labor Economics (IZA).
    4. Gary Charness & Thomas Garcia & Theo Offerman & Marie Claire Villeval, 2020. "Do measures of risk attitude in the laboratory predict behavior under risk in and outside of the laboratory?," Journal of Risk and Uncertainty, Springer, vol. 60(2), pages 99-123, April.
    5. Bittner, Anika & Haering, Alexander & Heinrich, Timo & Mayrhofer, Thomas, 2024. "Revalidating a survey instrument for measuring risk preferences," Economics Letters, Elsevier, vol. 242(C).
    6. Bühler, Dorothee & Sharma, Rasadhika & Stein, Wiebke, 2020. "Occupational Attainment and Earnings in Southeast Asia: The Role of Non-cognitive Skills," Labour Economics, Elsevier, vol. 67(C).
    7. Joshua Tasoff & Wenjie Zhang, 2022. "The Performance of Time-Preference and Risk-Preference Measures in Surveys," Management Science, INFORMS, vol. 68(2), pages 1149-1173, February.
    8. Müller, Stephan & Rau, Holger A., 2021. "Economic preferences and compliance in the social stress test of the COVID-19 crisis," Journal of Public Economics, Elsevier, vol. 194(C).
    9. Angerer, Silvia & Baier, Helena Antonie & Glätzle-Rützler, Daniela & Lergetporer, Philipp & Rittmannsberger, Thomas, 2024. "Economic Preferences Predict COVID-19 Vaccination Intentions and Behavior," IZA Discussion Papers 17533, Institute of Labor Economics (IZA).
    10. Müller, Stephan & Rau, Holger A., 2020. "Economic preferences and compliance in the social stress test of the Corona crisis," University of Göttingen Working Papers in Economics 391, University of Goettingen, Department of Economics.
    11. Armin Falk & Anke Becker & Thomas Dohmen & Benjamin Enke & David Huffman & Uwe Sunde, 2018. "Global Evidence on Economic Preferences," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 133(4), pages 1645-1692.
    12. Breitkopf, Laura & Chowdhury, Shyamal K. & Priyam, Shambhavi & Schildberg-Hörisch, Hannah & Sutter, Matthias, 2020. "Do economic preferences of children predict behavior?," DICE Discussion Papers 342, Heinrich Heine University Düsseldorf, Düsseldorf Institute for Competition Economics (DICE).
    13. Pablo Brañas-Garza & Diego Jorrat & Antonio M. Espín & Angel Sánchez, 2023. "Paid and hypothetical time preferences are the same: lab, field and online evidence," Experimental Economics, Springer;Economic Science Association, vol. 26(2), pages 412-434, April.
    14. Silvia Angerer & Helena Antonie Baier & Daniela Glätzle-Rützler & Philipp Lergetporer & Thomas Rittmannsberger, 2024. "Economic Preferences Predict Covid-19 Vaccination Intentions and Behavior," CESifo Working Paper Series 11566, CESifo.
    15. Silvia Angerer & Helena Antonie Baier & Daniela Glätzle-Rützler & Philipp Lergetporer & Thomas Rittmansberger, 2025. "Economic Preferences Predict COVID-19 Vaccination Intentions and Behavior," Munich Papers in Political Economy 37, Munich School of Politics and Public Policy and the School of Management at the Technical University of Munich.
    16. Dohmen, Thomas & Quercia, Simone & Willrodt, Jana, 2018. "Willingness to Take Risk: The Role of Risk Conception and Optimism," IZA Discussion Papers 11642, Institute of Labor Economics (IZA).
    17. Dohmen, Thomas & Lehmann, Hartmut & Pignatti, Norberto, 2016. "Time-varying individual risk attitudes over the Great Recession: A comparison of Germany and Ukraine," Journal of Comparative Economics, Elsevier, vol. 44(1), pages 182-200.
    18. Gill, Andrej & Hett, Florian & Tischer, Johannes, 2022. "Time inconsistency and overdraft use: Evidence from transaction data and behavioral measurement experiments," Discussion Papers 18/2022, Deutsche Bundesbank.
    19. Carole Treibich, 2015. "Are Survey Risk Aversion Measurements Adequate in a Low Income Context?," AMSE Working Papers 1517, Aix-Marseille School of Economics, France.
    20. Carpenter, Jeffrey, 2021. "The shape of warm glow: Field experimental evidence from a fundraiser," Journal of Economic Behavior & Organization, Elsevier, vol. 191(C), pages 555-574.

    More about this item

    Keywords

    experiment; preference measurement; survey; validation;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • D90 - Microeconomics - - Micro-Based Behavioral Economics - - - General

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp12450. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.