IDEAS home Printed from https://ideas.repec.org/a/plo/pone00/0258877.html
   My bibliography  Save this article

Representativeness of individual-level data in COVID-19 phone surveys: Findings from Sub-Saharan Africa

Author

Listed:
  • Joshua Brubaker
  • Talip Kilic
  • Philip Wollburg

Abstract

The COVID-19 pandemic has created urgent demand for timely data, leading to a surge in mobile phone surveys for tracking the impacts of and responses to the pandemic. Using data from national phone surveys implemented in Ethiopia, Malawi, Nigeria and Uganda during the pandemic and the pre-COVID-19 national face-to-face surveys that served as the sampling frames for the phone surveys, this paper documents selection the biases in individual-level analyses based on phone survey data. In most cases, individual-level data are available only for phone survey respondents, who we find are more likely to be household heads or their spouses and non-farm enterprise owners, and on average, are older and better educated vis-a-vis the general adult population. These differences are the result of uneven access to mobile phones in the population and the way that phone survey respondents are selected. To improve the representativeness of individual-level analysis using phone survey data, we recalibrate the phone survey sampling weights based on propensity score adjustments that are derived from a model of an individual’s likelihood of being interviewed as a function of individual- and household-level attributes. We find that reweighting improves the representativeness of the estimates for phone survey respondents, moving them closer to those of the general adult population. This holds for both women and men and for a range of demographic, education, and labor market outcomes. However, reweighting increases the variance of the estimates and, in most cases, fails to overcome selection biases. This indicates limitations to deriving representative individual-level estimates from phone survey data. Obtaining reliable data on men and women through future phone surveys will require random selection of adult interviewees within sampled households.

Suggested Citation

  • Joshua Brubaker & Talip Kilic & Philip Wollburg, 2021. "Representativeness of individual-level data in COVID-19 phone surveys: Findings from Sub-Saharan Africa," PLOS ONE, Public Library of Science, vol. 16(11), pages 1-27, November.
    • Handle: RePEc:plo:pone00:0258877
    DOI: 10.1371/journal.pone.0258877
    as

    Download full text from publisher

    File URL: https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0258877
    Download Restriction: no
    File URL: https://journals.plos.org/plosone/article/file?id=10.1371/journal.pone.0258877&type=printable
    Download Restriction: no
    File URL: https://libkey.io/10.1371/journal.pone.0258877?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Kanyanda,Shelton Sofiel Elisa & Markhof,Yannick Valentin & Wollburg,Philip Randolph & Zezza,Alberto, 2021. "Acceptance of COVID-19 Vaccines in Sub-Saharan Africa : Evidence from Six National Phone Surveys," Policy Research Working Paper Series 9739, The World Bank.
    2. Kilic, Talip & Moylan, Heather & Koolwal, Gayatri, 2021. "Getting the (Gender-Disaggregated) lay of the land: Impact of survey respondent selection on measuring land ownership and rights," World Development, Elsevier, vol. 146(C).
    3. Anna Josephson & Talip Kilic & Jeffrey D. Michler, 2021. "Socioeconomic impacts of COVID-19 in low-income countries," Nature Human Behaviour, Nature, vol. 5(5), pages 557-565, May.
    4. Ambel,Alemayehu A. & Mcgee,Kevin Robert & Tsegay,Asmelash Haile, 2021. "Reducing Bias in Phone Survey Samples : Effectiveness of Reweighting Techniques Using Face-to-Face Surveys as Frames in Four African Countries," Policy Research Working Paper Series 9676, The World Bank.
    5. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    6. Josephson,Anna Leigh & Kilic,Talip & Michler,Jeffrey David, 2020. "Socioeconomic Impacts of COVID-19 in Four African Countries," Policy Research Working Paper Series 9466, The World Bank.
    7. Kilic,Talip & Van den Broeck,Goedele & Koolwal,Gayatri B. & Moylan,Heather G., 2020. "Are You Being Asked ? Impacts of Respondent Selection on Measuring Employment," Policy Research Working Paper Series 9152, The World Bank.
    8. Melanie Khamis & Daniel Prinz & David Newhouse & Amparo Palacios-Lopez & Utz Pape & Michael Weber, 2021. "Early Labor Market Impacts of COVID-19 in Developing Countries," World Bank Publications - Reports 35047, The World Bank Group.
    9. Ben Leo, Robert Morello, Jonathan Mellon, Tiago Peixoto, and Stephen Davenport, 2015. "Do Mobile Phone Surveys Work in Poor Countries? - Working Paper 398," Working Papers 398, Center for Global Development.
    10. Furbush,Ann & Josephson,Anna Leigh & Kilic,Talip & Michler,Jeffrey David, 2021. "The Evolving Socioeconomic Impacts of COVID-19 in Four African Countries," Policy Research Working Paper Series 9556, The World Bank.
    11. Andrew Dabalen & Alvin Etang & Johannes Hoogeveen & Elvis Mushi & Youdi Schipper & Johannes von Engelhardt, 2016. "Mobile Phone Panel Surveys in Developing Countries," World Bank Publications - Books, The World Bank Group, number 24595.
    12. Kelly L’Engle & Eunice Sefa & Edward Akolgo Adimazoya & Emmanuel Yartey & Rachel Lenzi & Cindy Tarpo & Nii Lante Heward-Mills & Katherine Lew & Yvonne Ampeh, 2018. "Survey research with a random digit dial national mobile phone sample in Ghana: Methods and sample quality," PLOS ONE, Public Library of Science, vol. 13(1), pages 1-11, January.
    13. Quisumbing, Agnes R. & Kumar, Neha & Meinzen-Dick, Ruth Suseela & Ringler, Claudia, 2020. "Why gender matters in COVID-19 responses: Now and in the future," IFPRI book chapters, in: COVID-19 and global food security, chapter 20, pages 88-90, International Food Policy Research Institute (IFPRI).
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Margherita Squarcina & Eva-Maria Egger, 2022. "Effects of the COVID-19 crisis on household food consumption and child nutrition in Mozambique," WIDER Working Paper Series wp-2022-169, World Institute for Development Economic Research (UNU-WIDER).
    2. Tabakis,Chrysostomos & Ten,Gi Khan & Merfeld,Joshua David & Newhouse,David Locke & Pape,Utz Johann & Weber,Michael, 2022. "The Welfare Implications of COVID-19 for Fragile and Conflict-Affected Areas," Policy Research Working Paper Series 10081, The World Bank.
    3. Kugler, Maurice & Viollaz, Mariana & Duque, Daniel & Gaddis, Isis & Newhouse, David & Palacios-Lopez, Amparo & Weber, Michael, 2023. "How did the COVID-19 crisis affect different types of workers in the developing world?," World Development, Elsevier, vol. 170(C).
    4. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    5. Semakula, Henry Musoke & Liang, Song & Mukwaya, Paul Isolo & Mugagga, Frank, 2023. "Application of a Bayesian network modelling approach to predict the cascading effects of COVID-19 restrictions on the planting activities of smallholder farmers in Uganda," Agricultural Systems, Elsevier, vol. 211(C).
    6. Baumüller, Heike & Kornher, Lukas, 2024. "Inside the crowd: Assessing the suitability of SMSbased surveys to monitor the food security situation in Uganda," IAAE 2024 Conference, August 2-7, 2024, New Delhi, India 344389, International Association of Agricultural Economists (IAAE).
    7. Calogero Carletto, 2021. "Better data, higher impact: improving agricultural data systems for societal change [Correlated non-classical measurement errors, ‘second best’ policy inference, and the inverse size-productivity r," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 48(4), pages 719-740.
    8. Kanyanda,Shelton Sofiel Elisa & Markhof,Yannick Valentin & Wollburg,Philip Randolph & Zezza,Alberto, 2021. "Acceptance of COVID-19 Vaccines in Sub-Saharan Africa : Evidence from Six National Phone Surveys," Policy Research Working Paper Series 9739, The World Bank.
    9. Martin Paul Jr. Tabe-Ojong & Emmanuel Nshakira-Rukundo & Bisrat Haile Gebrekidan, 2023. "COVID-19 and food insecurity in Africa: A review of the emerging empirical evidence," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 50(3), pages 853-878.
    10. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gourlay, Sydney & Kilic, Talip & Martuscelli, Antonio & Wollburg, Philip & Zezza, Alberto, 2021. "Viewpoint: High-frequency phone surveys on COVID-19: Good practices, open questions," Food Policy, Elsevier, vol. 105(C).
    2. Monteiro Amaral,Sofia Fernando & Dinarte Diaz,Lelys Ileana & Dominguez,Patricio & Perez-Vincent,Santiago M. & Romero,Steffanny, 2022. "Talk or Text ? Evaluating Response Rates by Remote Survey Method during COVID-19," Policy Research Working Paper Series 9999, The World Bank.
    3. Anderson, Ellen & Lybbert, Travis J. & Shenoy, Ashish & Singh, Rupika & Stein, Daniel, 2024. "Does survey mode matter? Comparing in-person and phone agricultural surveys in India," Journal of Development Economics, Elsevier, vol. 166(C).
    4. Carletto,Calogero & Dillon,Andrew S. & Zezza,Alberto, 2021. "Agricultural Data Collection to Minimize Measurement Error and Maximize Coverage," Policy Research Working Paper Series 9745, The World Bank.
    5. Elisa M. Maffioli, 2020. "Collecting Data During an Epidemic: A Novel Mobile Phone Research Method," Journal of International Development, John Wiley & Sons, Ltd., vol. 32(8), pages 1231-1255, November.
    6. Calogero Carletto, 2021. "Better data, higher impact: improving agricultural data systems for societal change [Correlated non-classical measurement errors, ‘second best’ policy inference, and the inverse size-productivity r," European Review of Agricultural Economics, Oxford University Press and the European Agricultural and Applied Economics Publications Foundation, vol. 48(4), pages 719-740.
    7. Zezza,Alberto & Mcgee,Kevin Robert & Wollburg,Philip Randolph & Assefa,Thomas Woldu & Gourlay,Sydney, 2022. "From Necessity to Opportunity : Lessons for Integrating Phone and In-Person Data Collectionfor Agricultural Statistics in a Post-Pandemic World," Policy Research Working Paper Series 10168, The World Bank.
    8. Abate, Gashaw T. & de Brauw, Alan & Hirvonen, Kalle & Wolle, Abdulazize, 2023. "Measuring consumption over the phone: Evidence from a survey experiment in urban Ethiopia," Journal of Development Economics, Elsevier, vol. 161(C).
    9. Kugler, Maurice & Viollaz, Mariana & Duque, Daniel & Gaddis, Isis & Newhouse, David & Palacios-Lopez, Amparo & Weber, Michael, 2023. "How did the COVID-19 crisis affect different types of workers in the developing world?," World Development, Elsevier, vol. 170(C).
    10. Ambel, Alemayehu & McGee, Kevin & Tsegay, Asmelash, 2021. "Reducing Bias in Phone Survey Samples: Effectiveness of Reweighting Techniques Using Face-to-Face Surveys As Frames," 2021 Conference, August 17-31, 2021, Virtual 315301, International Association of Agricultural Economists.
    11. Ahmed Mushfiq Mobarak & Edward Miguel, 2022. "The Economics of the COVID-19 Pandemic in Poor Countries," Annual Review of Economics, Annual Reviews, vol. 14(1), pages 253-285, August.
    12. Joanna Upton & Elizabeth Tennant & Kathryn J. Fiorella & Christopher B. Barrett, 2023. "COVID-19, Household Resilience, and Rural Food Systems: Evidence from Southern and Eastern Africa," Palgrave Studies in Agricultural Economics and Food Policy, in: Christophe Béné & Stephen Devereux (ed.), Resilience and Food Security in a Food Systems Context, chapter 0, pages 281-320, Palgrave Macmillan.
    13. World Bank Group, 2016. "Republic of Guinea," World Bank Publications - Reports 24724, The World Bank Group.
    14. Abdullah Kaviani Rad & Redmond R. Shamshiri & Hassan Azarm & Siva K. Balasundram & Muhammad Sultan, 2021. "Effects of the COVID-19 Pandemic on Food Security and Agriculture in Iran: A Survey," Sustainability, MDPI, vol. 13(18), pages 1-20, September.
    15. Tilman Brück & Mekdim D. Regassa, 2023. "Usefulness and misrepresentation of phone surveys on COVID-19 and food security in Africa," Food Security: The Science, Sociology and Economics of Food Production and Access to Food, Springer;The International Society for Plant Pathology, vol. 15(2), pages 423-453, April.
    16. Kalle Hirvonen & Alan de Brauw & Gashaw T. Abate, 2021. "Food Consumption and Food Security during the COVID‐19 Pandemic in Addis Ababa," American Journal of Agricultural Economics, John Wiley & Sons, vol. 103(3), pages 772-789, May.
    17. Elena Serfilippi & Daniele Giovannucci & David Ameyaw & Ankur Bansal & Thomas Asafua Nketsia Wobill & Roberta Blankson & Rashi Mishra, 2022. "Benefits and Challenges of Making Data More Agile: A Review of Recent Key Approaches in Agriculture," Sustainability, MDPI, vol. 14(24), pages 1-18, December.
    18. Ndashimye, Felix & Hebie, Oumarou & Tjaden, Jasper, 2021. "Effectiveness of WhatsApp for measuring migration in follow-up phone surveys - Lessons from a mode experiment in two low-income countries during COVID contact restrictions," OSF Preprints khd32, Center for Open Science.
    19. Hammond, James & Siegal, Kim & Milner, Daniel & Elimu, Emmanuel & Vail, Taylor & Cathala, Paul & Gatera, Arsene & Karim, Azfar & Lee, Ja-Eun & Douxchamps, Sabine & Tu, Mai Thanh & Ouma, Emily & Lukuyu, 2022. "Perceived effects of COVID-19 restrictions on smallholder farmers: Evidence from seven lower- and middle-income countries," Agricultural Systems, Elsevier, vol. 198(C).
    20. Gern, Klaus-Jürgen & Lück, Ole & Meuchelböck, Saskia, 2021. "Covid-19 in Africa and its impact on the economy," Kiel Policy Brief 158, Kiel Institute for the World Economy (IfW Kiel).

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:plo:pone00:0258877. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: plosone (email available below). General contact details of provider: https://journals.plos.org/plosone/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.