IDEAS home Printed from https://ideas.repec.org/a/gam/jijerp/v18y2021i22p12181-d683719.html
   My bibliography  Save this article

Identifying and Minimizing Errors in the Measurement of Early Childhood Development: Lessons Learned from the Cognitive Testing of the ECDI2030

Author

Listed:
  • Claudia Cappa

    (United Nations Children’s Fund, New York, NY 10017, USA)

  • Nicole Petrowski

    (United Nations Children’s Fund, New York, NY 10017, USA)

  • Elga Filipa De Castro

    (United Nations Children’s Fund, New York, NY 10017, USA)

  • Emily Geisen

    (Qualtrics, Raleigh, NC 27603, USA)

  • Patricia LeBaron

    (RTI International, Research Triangle Park, NC 27709, USA)

  • Betania Allen-Leigh

    (Instituto Nacional de Salud Pública, Cuernavaca, Morelos 62100, Mexico)

  • Jean Marie Place

    (Department of Nutrition and Health Science, Ball State University, Muncie, IN 47303, USA)

  • Paul J. Scanlon

    (National Center for Health Statistics, Hyattsville, MD 20782, USA)

Abstract

Challenges in measuring early childhood development (ECD) at scale have been documented, yet little is known about the specific difficulties related to questionnaire design and question interpretation. The purpose of this paper is to discuss the challenges of measuring ECD at scale in the context of household surveys and to show how to overcome them. The paper uses examples from the cognitive interviewing exercises that were conducted as part of the methodological work to develop a measure of ECD outcomes, the ECDI2030. It describes the methodological work carried out to inform the selection and improvement of question items and survey implementation tools as a fundamental step to reduce and mitigate systematic measurement error and improve data quality. The project consisted of a total of five rounds of testing, comprising 191 one-on-one, in-depth cognitive interviews across six countries (Bulgaria, India, Jamaica, Mexico, Uganda, and the USA). Qualitative data analysis methods were used to determine matches and mismatches between intention of items and false positives or false negative answers among subgroups of respondents. Key themes emerged that could potentially lead to systematic measurement error in population-based surveys on ECD: (1) willingness of child to perform task versus ability of child to perform task; (2) performing task versus performing task correctly; (3) identifying letters or numbers versus recognizing letters or numbers; (4) consistently performing task versus correctly performing task; (5) applicability of skills being asked versus observability of skills being asked; and (6) language production versus language comprehension. Through an iterative process of testing and subsequent revision, improvements were made to item wording, response options, and interviewer training instructions. Given the difficulties inherent in population-level data collection in the context of global monitoring, this study’s findings confirm the importance of cognitive testing as a crucial step in careful, culturally relevant, and sensitive questionnaire design and as a means to reduce response bias in cross-cultural contexts.

Suggested Citation

  • Claudia Cappa & Nicole Petrowski & Elga Filipa De Castro & Emily Geisen & Patricia LeBaron & Betania Allen-Leigh & Jean Marie Place & Paul J. Scanlon, 2021. "Identifying and Minimizing Errors in the Measurement of Early Childhood Development: Lessons Learned from the Cognitive Testing of the ECDI2030," IJERPH, MDPI, vol. 18(22), pages 1-13, November.
  • Handle: RePEc:gam:jijerp:v:18:y:2021:i:22:p:12181-:d:683719
    as

    Download full text from publisher

    File URL: https://www.mdpi.com/1660-4601/18/22/12181/pdf
    Download Restriction: no

    File URL: https://www.mdpi.com/1660-4601/18/22/12181/
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Lia C. H. Fernald & Elizabeth Prado & Patricia Kariger & Abbie Raikes, 2017. "A Toolkit for Measuring Early Childhood Development in Low and Middle-Income Countries," World Bank Publications - Books, The World Bank Group, number 29000.
    2. Dana Charles McCoy & Evan D Peet & Majid Ezzati & Goodarz Danaei & Maureen M Black & Christopher R Sudfeld & Wafaie Fawzi & Günther Fink, 2016. "Early Childhood Developmental Status in Low- and Middle-Income Countries: National, Regional, and Global Prevalence Estimates Using Predictive Modeling," PLOS Medicine, Public Library of Science, vol. 13(6), pages 1-18, June.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kasim Allel & Gerard Abou Jaoude & Stavros Poupakis & Neha Batura & Jolene Skordis & Hassan Haghparast-Bidgoli, 2021. "Exploring the Associations between Early Childhood Development Outcomes and Ecological Country-Level Factors across Low- and Middle-Income Countries," IJERPH, MDPI, vol. 18(7), pages 1-15, March.
    2. Holla,Alaka & Bendini,Maria Magdalena & Dinarte Diaz,Lelys Ileana & Trako,Iva, 2021. "Is Investment in Preprimary Education Too Low ? Lessons from (Quasi) ExperimentalEvidence across Countries," Policy Research Working Paper Series 9723, The World Bank.
    3. Orazio Attanasio & Helen Baker-Henningham & Raquel Bernal & Costas Meghir & Diana Pineda & Marta Rubio-Codina, 2022. "Early Stimulation and Nutrition: The Impacts of a Scalable Intervention," Journal of the European Economic Association, European Economic Association, vol. 20(4), pages 1395-1432.
    4. Lebedinski, Lara & Carneiro, Pedro & Urzua, Tamara Arnold & Perng, Julie & Boudet, Ana Maria Munoz & Sosa, Katia Herrera, 2024. "Experimental Impacts of a Virtual Parenting Program with Mothers and Fathers," IZA Discussion Papers 17028, Institute of Labor Economics (IZA).
    5. Özler, Berk & Fernald, Lia C.H. & Kariger, Patricia & McConnell, Christin & Neuman, Michelle & Fraga, Eduardo, 2018. "Combining pre-school teacher training with parenting education: A cluster-randomized controlled trial," Journal of Development Economics, Elsevier, vol. 133(C), pages 448-467.
    6. Jorge Luis García & James J. Heckman, 2022. "Parenting Promotes Social Mobility Within and Across Generations," NBER Working Papers 30610, National Bureau of Economic Research, Inc.
    7. Jorge Luis Garcia & James J. Heckman, 2024. ""Unsettled Science on Longer-run Effects of Early Education:" A Response," Working Papers 2024-012, Human Capital and Economic Opportunity Working Group.
    8. Buck,Lindsey & Fiala,Nathan V. & Prakash,Nishith & Sabarwal,Shwetlena & Saraswat,Deepak & Shrestha,Deepika, 2020. "Educator Knowledge of Early Childhood Development : Evidence from Eastern Nepal," Policy Research Working Paper Series 9381, The World Bank.
    9. Bago, Jean-Louis & Ouédraogo, Moussa & Akakpo, Koffi & Lompo, Miaba Louise & Souratié, Wamadini M. & Ouédraogo, Ernest, 2019. "Early Childhood Education and Children Development : Evidence from Ghana," MPRA Paper 95868, University Library of Munich, Germany.
    10. Akgündüz, Yusuf Emre & Akyol, Pelin & Aydemir, Abdurrahman B. & Demirci, Murat & Kirdar, Murat Güray, 2023. "Maternal Education and Early Child Development: The Roles of Parental Support for Learning, Learning Materials, and Father Characteristics," IZA Discussion Papers 16328, Institute of Labor Economics (IZA).
    11. Fink, Günther & Venkataramani, Atheendar S. & Zanolini, Arianna, 2021. "Early life adversity, biological adaptation, and human capital: evidence from an interrupted malaria control program in Zambia," Journal of Health Economics, Elsevier, vol. 80(C).
    12. Orazio Attanasio & Raquel Bernal & Michele Giannola & Milagros Nores, 2021. "Child Development in the Early Years: Parental Investment and the Changing Dynamics of Different Domains," CSEF Working Papers 626, Centre for Studies in Economics and Finance (CSEF), University of Naples, Italy.
    13. Nores, Milagros & Bernal, Raquel & Barnett, W. Steven, 2019. "Center-based care for infants and toddlers: The aeioTU randomized trial," Economics of Education Review, Elsevier, vol. 72(C), pages 30-43.
    14. Dejana Bouillet & Sandra Antulić Majcen, 2022. "Risks of Social Exclusion Among Children in ECEC Settings: Assessments by Parents and ECEC Teachers," SAGE Open, , vol. 12(3), pages 21582440221, September.
    15. Xu, Yanfeng & Man, Xiaoou & Zhang, Lixia & DeForge, Bruce, 2020. "Family foster care and children’s outcomes in China: Evidence from a scoping review," Children and Youth Services Review, Elsevier, vol. 108(C).
    16. Raquel Bernal & Michele Giannola & Milagros Nores, 2022. "The Effect of Center-Based Early Education on Disadvantaged Children's Developmental Trajectories: Experimental Evidence from Colombia," Working Papers 2022-027, Human Capital and Economic Opportunity Working Group.
    17. Hannah Johnstone & Yi Yang & Hao Xue & Scott Rozelle, 2021. "Infant Cognitive Development and Stimulating Parenting Practices in Rural China," IJERPH, MDPI, vol. 18(10), pages 1-16, May.
    18. Verónica Schiariti & Rune J. Simeonsson & Karen Hall, 2021. "Promoting Developmental Potential in Early Childhood: A Global Framework for Health and Education," IJERPH, MDPI, vol. 18(4), pages 1-15, February.
    19. Sue Woolfenden & Kate Milner & Kali Tora & Kelera Naulumatua & Reapi Mataika & Fleur Smith & Raghu Lingam & Joseph Kado & Ilisapeci Tuibeqa, 2020. "Strengthening Health Systems to Support Children with Neurodevelopmental Disabilities in Fiji—A Commentary," IJERPH, MDPI, vol. 17(3), pages 1-11, February.
    20. Bago, Jean-Louis & Ouédraogo, Moussa & Akakpo, Koffi & Lompo, Miaba Louise & Souratié, Wamadini dite Minata & Ouédraogo, Ernest, 2020. "Early Childhood Education and Child Development: New Evidence from Ghana," Children and Youth Services Review, Elsevier, vol. 108(C).

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:gam:jijerp:v:18:y:2021:i:22:p:12181-:d:683719. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: MDPI Indexing Manager (email available below). General contact details of provider: https://www.mdpi.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.