IDEAS home Printed from https://ideas.repec.org/a/eee/deveco/v168y2024ics0304387824000142.html
   My bibliography  Save this article

Whom to ask? Testing respondent effects in household surveys

Author

Listed:
  • Masselus, Lise
  • Fiala, Nathan

Abstract

Household questionnaires typically survey the most knowledgeable household member, but this can lead to inaccurate data if they have limited information. Using data from survey experiments with 4,100 households in Paraguay and Uganda, we investigate whether there are discrepancies in intra-household reporting when multiple household members are interviewed. We randomly vary who responds to a survey on household income and food consumption using common approaches to respondent selection. We find that the mean and distribution of these variables is insensitive to respondent selection. However, there are discrepancies between spouses of the same household in Uganda that depend on the gender of the respondent or recipient of the income. Taken together, our results indicate that respondent selection does not markedly affect the aggregate analysis of households, but that it may matter for obtaining accurate information on income and consumption for a given household or by gender.

Suggested Citation

  • Masselus, Lise & Fiala, Nathan, 2024. "Whom to ask? Testing respondent effects in household surveys," Journal of Development Economics, Elsevier, vol. 168(C).
    • Handle: RePEc:eee:deveco:v:168:y:2024:i:c:s0304387824000142
    DOI: 10.1016/j.jdeveco.2024.103265
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0304387824000142
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.jdeveco.2024.103265?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Beaman, Lori & Dillon, Andrew, 2012. "Do household definitions matter in survey design? Results from a randomized survey experiment in Mali," Journal of Development Economics, Elsevier, vol. 98(1), pages 124-135.
    2. de Mel, Suresh & McKenzie, David J. & Woodruff, Christopher, 2009. "Measuring microenterprise profits: Must we ask how the sausage is made?," Journal of Development Economics, Elsevier, vol. 88(1), pages 19-31, January.
    3. Talip Kilic & Thomas Pave Sohnesen, 2019. "Same Question But Different Answer: Experimental Evidence on Questionnaire Design's Impact on Poverty Measured by Proxies," Review of Income and Wealth, International Association for Research in Income and Wealth, vol. 65(1), pages 144-165, March.
    4. Doss, Cheryl R. & Catanzarite, Zachary & Baah-Boateng, William & Swaminathan, Hema & Diana Deere, Carmen & Boakye-Yiadom, Louis & Suchitra, J.Y., 2018. "Do men and women estimate property values differently?," World Development, Elsevier, vol. 107(C), pages 75-86.
    5. Rachid Laaja & Karen Macours, 2021. "Measuring Skills in Developing Countries," Journal of Human Resources, University of Wisconsin Press, vol. 56(4), pages 1254-1295.
    6. Fisher, Monica & Reimer, Jeffrey J. & Carr, Edward R., 2010. "Who Should be Interviewed in Surveys of Household Income?," World Development, Elsevier, vol. 38(7), pages 966-973, July.
    7. Christopher Blattman & Nathan Fiala & Sebastian Martinez, 2014. "Generating Skilled Self-Employment in Developing Countries: Experimental Evidence from Uganda," The Quarterly Journal of Economics, President and Fellows of Harvard College, vol. 129(2), pages 697-752.
    8. Dillon, Andrew & Bardasi, Elena & Beegle, Kathleen & Serneels, Pieter, 2012. "Explaining variation in child labor statistics," Journal of Development Economics, Elsevier, vol. 98(1), pages 136-147.
    9. Beegle, Kathleen & De Weerdt, Joachim & Friedman, Jed & Gibson, John, 2012. "Methods of household consumption measurement through surveys: Experimental results from Tanzania," Journal of Development Economics, Elsevier, vol. 98(1), pages 3-18.
    10. Kate Ambler & Cheryl Doss & Caitlin Kieran & Simone Passarelli, 2021. "He Says, She Says: Spousal Disagreement in Survey Measures of Bargaining Power," Economic Development and Cultural Change, University of Chicago Press, vol. 69(2), pages 765-788.
    11. Beegle, Kathleen & Carletto, Calogero & Himelein, Kristen, 2012. "Reliability of recall in agricultural data," Journal of Development Economics, Elsevier, vol. 98(1), pages 34-41.
    12. Christopher Blattman & Nathan Fiala & Sebastian Martinez, 2020. "The Long-Term Impacts of Grants on Poverty: Nine-Year Evidence from Uganda's Youth Opportunities Program," American Economic Review: Insights, American Economic Association, vol. 2(3), pages 287-304, September.
    13. Dammert, Ana C. & Galdo, Jose, 2021. "Assessing adult farm labor statistics: Evidence from a survey design experiment in Ethiopia," Economics Letters, Elsevier, vol. 203(C).
    14. Serneels, Pieter & Beegle, Kathleen & Dillon, Andrew, 2017. "Do returns to education depend on how and whom you ask?," Economics of Education Review, Elsevier, vol. 60(C), pages 5-19.
    15. Jose Galdo & Ana C Dammert & Degnet Abebaw, 2021. "Gender Bias in Agricultural Child Labor: Evidence from Survey Design Experiments," The World Bank Economic Review, World Bank, vol. 35(4), pages 872-891.
    16. Kilic, Talip & Moylan, Heather & Koolwal, Gayatri, 2021. "Getting the (Gender-Disaggregated) lay of the land: Impact of survey respondent selection on measuring land ownership and rights," World Development, Elsevier, vol. 146(C).
    17. Desiere,Sam & Costa,Valentina, 2019. "Employment Data in Household Surveys : Taking Stock, Looking Ahead," Policy Research Working Paper Series 8882, The World Bank.
    18. Fletschner, Diana & Mesbah, Dina, 2011. "Gender Disparity in Access to Information: Do Spouses Share What They Know?," World Development, Elsevier, vol. 39(8), pages 1422-1433, August.
    19. de Nicola, Francesca & Giné, Xavier, 2014. "How accurate are recall data? Evidence from coastal India," Journal of Development Economics, Elsevier, vol. 106(C), pages 52-65.
    20. repec:oup:qjecon:v:129:y:2014:i:2:p:697-752. is not listed on IDEAS
    21. Nillesen, Eleonora & Grimm, Michael & Goedhuys, Micheline & Reitmann, Ann-Kristin & Meysonnat, Aline, 2021. "On the malleability of gender attitudes: Evidence from implicit and explicit measures in Tunisia," World Development, Elsevier, vol. 138(C).
    22. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    23. Ambler, Kate & Herskowitz, Sylvan & Maredia, Mywish K., 2021. "Are we done yet? Response fatigue and rural livelihoods," Journal of Development Economics, Elsevier, vol. 153(C).
    24. Joachim De Weerdt & John Gibson & Kathleen Beegle, 2020. "What Can We Learn from Experimenting with Survey Methods?," Annual Review of Resource Economics, Annual Reviews, vol. 12(1), pages 431-447, October.
    25. Dillon, Andrew & Karlan, Dean & Udry, Christopher & Zinman, Jonathan, 2020. "Good identification, meet good data," World Development, Elsevier, vol. 127(C).
    26. Talip Kilic & Goedele Van den & Gayatri Koolwal & Heather Moylan, 2023. "Are You Being Asked? Impacts of Respondent Selection on Measuring Employment in Malawi," Journal of African Economies, Centre for the Study of African Economies, vol. 32(5), pages 495-522.
    27. Gero Carletto & Marco Tiberti & Alberto Zezza, 2022. "Measure for Measure: Comparing Survey Based Estimates of Income and Consumption for Rural Households [Obtaining Useful Data on Household Incomes from Surveys]," The World Bank Research Observer, World Bank, vol. 37(1), pages 1-38.
    28. Talip Kilic & Heather Moylan, 2016. "Methodological Experiment on Measuring Asset Ownership from a Gender Perspective," World Bank Publications - Reports 33653, The World Bank Group.
    29. Robert Garlick & Kate Orkin & Simon Quinn, 2020. "Call Me Maybe: Experimental Evidence on Frequency and Medium Effects in Microenterprise Surveys," The World Bank Economic Review, World Bank, vol. 34(2), pages 418-443.
    30. Rachel Heath & Ghazala Mansuri & Bob Rijkers & William Seitz & Dhiraj Sharma, 2021. "Measuring Employment: Experimental Evidence from Urban Ghana," The World Bank Economic Review, World Bank, vol. 35(3), pages 635-651.
    31. Friedman, Jed & Beegle, Kathleen & De Weerdt, Joachim & Gibson, John, 2017. "Decomposing response error in food consumption measurement: Implications for survey design from a randomized survey experiment in Tanzania," Food Policy, Elsevier, vol. 72(C), pages 94-111.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Beegle, Kathleen & Dillon, Andrew & Karlan, Dean & Udry, Christopher, 2024. "Introduction to the journal of development economics special issue on methods and measurement," Journal of Development Economics, Elsevier, vol. 170(C).

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Fiala, Nathan & Masselus, Lise, 2022. "Whom to ask? Testing respondent effects in household surveys," Ruhr Economic Papers 935, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    2. repec:lic:licosd:41819 is not listed on IDEAS
    3. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    4. Ambler, Kate & Herskowitz, Sylvan & Maredia, Mywish K., 2021. "Are we done yet? Response fatigue and rural livelihoods," Journal of Development Economics, Elsevier, vol. 153(C).
    5. Fiala, Nathan & Rose, Julian & Aryemo, Filder & Peters, Jörg, 2022. "The (very) long-run impacts of cash grants during a crisis," Ruhr Economic Papers 961, RWI - Leibniz-Institut für Wirtschaftsforschung, Ruhr-University Bochum, TU Dortmund University, University of Duisburg-Essen.
    6. Fitzpatrick, Anne, 2023. "Which price is right? A comparison of three standard approaches to measuring prices," Journal of Development Economics, Elsevier, vol. 163(C).
    7. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    8. Gourlay,Sydney & Maggio,Giuseppe & Safyan,Anahit & Zezza,Alberto, 2022. "Measuring Land Tenure at the Individual Level : Lessons from Methodological Research in Armenia," Policy Research Working Paper Series 10140, The World Bank.
    9. Dillon, Andrew & Mensah, Edouard, 2024. "Respondent biases in agricultural household surveys," Journal of Development Economics, Elsevier, vol. 166(C).
    10. Cheryl R. Doss & Agnes R. Quisumbing, 2020. "Understanding rural household behavior: Beyond Boserup and Becker," Agricultural Economics, International Association of Agricultural Economists, vol. 51(1), pages 47-58, January.
    11. Koolwal, Gayatri B., 2021. "Improving the measurement of rural women's employment: Global momentum and survey priorities," World Development, Elsevier, vol. 147(C).
    12. Zezza, Alberto & Federighi, Giovanni & Kalilou, Amadou Adamou & Hiernaux, Pierre, 2016. "Milking the data: Measuring milk off-take in extensive livestock systems. Experimental evidence from Niger," Food Policy, Elsevier, vol. 59(C), pages 174-186.
    13. Monteiro Amaral,Sofia Fernando & Dinarte Diaz,Lelys Ileana & Dominguez,Patricio & Perez-Vincent,Santiago M. & Romero,Steffanny, 2022. "Talk or Text ? Evaluating Response Rates by Remote Survey Method during COVID-19," Policy Research Working Paper Series 9999, The World Bank.
    14. Genicot, Garance & Hernandez-de-Benito, Maria, 2022. "Women’s land rights and village institutions in Tanzania," World Development, Elsevier, vol. 153(C).
    15. Kalaj, Jozefina & Rogger, Daniel & Somani, Ravi, 2022. "Bureaucrat time-use: Evidence from a survey experiment," World Development, Elsevier, vol. 152(C).
    16. Arthi, Vellore & Beegle, Kathleen & De Weerdt, Joachim & Palacios-López, Amparo, 2018. "Not your average job: Measuring farm labor in Tanzania," Journal of Development Economics, Elsevier, vol. 130(C), pages 160-172.
    17. Abay, Kibrom A. & Berhane, Guush & Hoddinott, John F. & Tafere, Kibrom, 2021. "Assessing response fatigue in phone surveys: Experimental evidence on dietary diversity in Ethiopia," IFPRI discussion papers 2017, International Food Policy Research Institute (IFPRI).
    18. Dervisevic, Ervin & Goldstein, Markus, 2023. "He said, she said: The impact of gender and marriage perceptions on self and proxy reporting of labor," Journal of Development Economics, Elsevier, vol. 161(C).
    19. Carletto, Calogero & Gourlay, Sydney & Murray, Siobhan & Zezza, Alberto, 2015. "Welcome to Fantasyland: Comparing Approaches To Land Area Measurement In Household Surveys," 2015 Conference, August 9-14, 2015, Milan, Italy 211849, International Association of Agricultural Economists.
    20. Godlonton, Susan & Hernandez, Manuel A. & Paz, Cynthia, 2021. "Can survey design reduce anchoring bias in recall data? Evidence from Malawi," IFPRI discussion papers 2055, International Food Policy Research Institute (IFPRI).
    21. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.

    More about this item

    Keywords

    Survey methods; Respondent effects; Proxy reporting; Intra-household economics;
    All these keywords.

    JEL classification:

    • O1 - Economic Development, Innovation, Technological Change, and Growth - - Economic Development
    • C8 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs
    • J16 - Labor and Demographic Economics - - Demographic Economics - - - Economics of Gender; Non-labor Discrimination
    • D13 - Microeconomics - - Household Behavior - - - Household Production and Intrahouse Allocation
    • I32 - Health, Education, and Welfare - - Welfare, Well-Being, and Poverty - - - Measurement and Analysis of Poverty

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:deveco:v:168:y:2024:i:c:s0304387824000142. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/devec .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.