IDEAS home Printed from https://ideas.repec.org/a/vrs/offsta/v33y2017i3p753-779n9.html
   My bibliography  Save this article

Using Response Propensity Models to Improve the Quality of Response Data in Longitudinal Studies

Author

Listed:
  • Plewis Ian

    (Social Statistics, School of Social Sciences, University of Manchester, ManchesterM13 9PL, United Kingdom of Great Britain and Northern Ireland.)

  • Shlomo Natalie

    (Social Statistics, School of Social Sciences, University of Manchester, ManchesterM13 9PL, United Kingdom of Great Britain and Northern Ireland.)

Abstract

We review two approaches for improving the response in longitudinal (birth cohort) studies based on response propensity models: strategies for sample maintenance in longitudinal studies and improving the representativeness of the respondents over time through interventions. Based on estimated response propensities, we examine the effectiveness of different re-issuing strategies using Representativity Indicators (R-indicators). We also combine information from the Receiver Operating Characteristic (ROC) curve with a cost function to determine an optimal cut point for the propensity not to respond in order to target interventions efficiently at cases least likely to respond. We use the first four waves of the UK Millennium Cohort Study to illustrate these methods. Our results suggest that it is worth re-issuing to the field nonresponding cases from previous waves although re-issuing refusals might not be the best use of resources. Adapting the sample to target subgroups for re-issuing from wave to wave will improve the representativeness of response. However, in situations where discrimination between respondents and nonrespondents is not strong, it is doubtful whether specific interventions to reduce nonresponse will be cost effective.

Suggested Citation

  • Plewis Ian & Shlomo Natalie, 2017. "Using Response Propensity Models to Improve the Quality of Response Data in Longitudinal Studies," Journal of Official Statistics, Sciendo, vol. 33(3), pages 753-779, September.
    • Handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:753-779:n:9
    DOI: 10.1515/jos-2017-0035
    as

    Download full text from publisher

    File URL: https://doi.org/10.1515/jos-2017-0035
    Download Restriction: no
    File URL: https://libkey.io/10.1515/jos-2017-0035?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Gabriele B. Durrant & Julia D'Arrigo & Fiona Steele, 2011. "Using paradata to predict best times of contact, conditioning on household and interviewer influences," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 174(4), pages 1029-1049, October.
    2. F. Kreuter & K. Olson & J. Wagner & T. Yan & T. M. Ezzati‐Rice & C. Casas‐Cordero & M. Lemay & A. Peytchev & R. M. Groves & T. E. Raghunathan, 2010. "Using proxy measures and other correlates of survey outcomes to adjust for non‐response: examples from multiple surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 173(2), pages 389-407, April.
    3. Durrant, Gabriele B. & Steele, Fiona, 2009. "Multilevel modelling of refusal and non-contact in household surveys: evidence from six UK Government surveys," LSE Research Online Documents on Economics 50112, London School of Economics and Political Science, LSE Library.
    4. Gabriele B. Durrant & Fiona Steele, 2009. "Multilevel modelling of refusal and non‐contact in household surveys: evidence from six UK Government surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(2), pages 361-381, April.
    5. Laura Fumagalli & Heather Laurie & Peter Lynn, 2013. "Experiments with methods to reduce attrition in longitudinal surveys," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 176(2), pages 499-519, February.
    6. Skinner, Chris J. & D'Arrigo, Julia, 2011. "Inverse probability weighting for clustered nonresponse," LSE Research Online Documents on Economics 40308, London School of Economics and Political Science, LSE Library.
    7. C. J. Skinner & D'arrigo, 2011. "Inverse probability weighting for clustered nonresponse," Biometrika, Biometrika Trust, vol. 98(4), pages 953-966.
    8. Watson, Nicole & Pudney, Stephen, 2013. "If at first you don’t succeed? Fieldwork, panel attrition, and health-employment inferences in BHPS and HILDA," ISER Working Paper Series 2013-27, Institute for Social and Economic Research.
    9. Barry Schouten & Natalie Shlomo, 2017. "Selecting Adaptive Survey Design Strata with Partial R-indicators," International Statistical Review, International Statistical Institute, vol. 85(1), pages 143-163, April.
    10. William J. Browne & Fiona Steele & Mousa Golalizadeh & Martin J. Green, 2009. "The use of simple reparameterizations to improve the efficiency of Markov chain Monte Carlo estimation for multilevel models with applications to discrete time survival models," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(3), pages 579-598, June.
    11. Schouten, Barry & Shlomo, Natalie & Skinner, Chris J., 2011. "Indicators for monitoring and improving representativeness of response," LSE Research Online Documents on Economics 39121, London School of Economics and Political Science, LSE Library.
    12. Lynn Peter & Kaminska Olena & Goldstein Harvey, 2014. "Panel Attrition: How Important is Interviewer Continuity?," Journal of Official Statistics, Sciendo, vol. 30(3), pages 443-457, September.
    13. Nicole Watson & Mark Wooden, 2014. "Re-engaging with survey non-respondents: evidence from three household panels," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 177(2), pages 499-522, February.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Gabriele B. Durrant & Sylke V. Schnepf, 2018. "Which schools and pupils respond to educational achievement surveys?: a focus on the English Programme for International Student Assessment sample," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1057-1075, October.
    2. Brick J. Michael, 2013. "Unit Nonresponse and Weighting Adjustments: A Critical Review," Journal of Official Statistics, Sciendo, vol. 29(3), pages 329-353, June.
    3. Durrant Gabriele B. & Maslovskaya Olga & Smith Peter W. F., 2017. "Using Prior Wave Information and Paradata: Can They Help to Predict Response Outcomes and Call Sequence Length in a Longitudinal Study?," Journal of Official Statistics, Sciendo, vol. 33(3), pages 801-833, September.
    4. Steele, Fiona & Durrant, Gabriele B., 2011. "Alternative approaches to multilevel modelling of survey non-contact and refusal," LSE Research Online Documents on Economics 50113, London School of Economics and Political Science, LSE Library.
    5. Kristen Olson, 2013. "Paradata for Nonresponse Adjustment," The ANNALS of the American Academy of Political and Social Science, , vol. 645(1), pages 142-170, January.
    6. Jamie C. Moore & Gabriele B. Durrant & Peter W. F. Smith, 2021. "Do coefficients of variation of response propensities approximate non‐response biases during survey data collection?," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(1), pages 301-323, January.
    7. Alireza Rezaee & Mojtaba Ganjali & Ehsan Bahrami Samani, 2022. "Sample selection bias with multiple dependent selection rules: an application to survey data analysis with multilevel nonresponse," Swiss Journal of Economics and Statistics, Springer;Swiss Society of Economics and Statistics, vol. 158(1), pages 1-15, December.
    8. Roger Tourangeau & J. Michael Brick & Sharon Lohr & Jane Li, 2017. "Adaptive and responsive survey designs: a review and assessment," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 180(1), pages 203-223, January.
    9. Roberts Caroline & Vandenplas Caroline & Herzing Jessica M.E., 2020. "A Validation of R-Indicators as a Measure of the Risk of Bias using Data from a Nonresponse Follow-Up Survey," Journal of Official Statistics, Sciendo, vol. 36(3), pages 675-701, September.
    10. Jouni Kuha & Myrsini Katsikatsou & Irini Moustaki, 2018. "Latent variable modelling with non‐ignorable item non‐response: multigroup response propensity models for cross‐national analysis," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 181(4), pages 1169-1192, October.
    11. Wagner James & Olson Kristen, 2018. "An Analysis of Interviewer Travel and Field Outcomes in Two Field Surveys," Journal of Official Statistics, Sciendo, vol. 34(1), pages 211-237, March.
    12. Michele Lalla & Maddalena Cavicchioli, 2020. "Nonresponse and measurement errors in income: matching individual survey data with administrative tax data," Department of Economics 0170, University of Modena and Reggio E., Faculty of Economics "Marco Biagi".
    13. Mark Amos, 2018. "Interviewer effects on patterns of nonresponse: Evaluating the impact on the reasons for contraceptive nonuse in the Indonesia and the Philippines DHS," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 39(14), pages 415-430.
    14. Finaba Berete & Johan Van der Heyden & Stefaan Demarest & Rana Charafeddine & Lydia Gisle & Elise Braekman & Jean Tafforeau & Geert Molenberghs, 2019. "Determinants of unit nonresponse in multi-mode data collection: A multilevel analysis," PLOS ONE, Public Library of Science, vol. 14(4), pages 1-18, April.
    15. Schnepf, Sylke V. & Durrant, Gabriele B. & Micklewright, John, 2014. "Which Schools and Pupils Respond to Educational Achievement Surveys? A Focus on the English PISA Sample," IZA Discussion Papers 8411, Institute of Labor Economics (IZA).
    16. Earp Morgan & Toth Daniell & Phipps Polly & Oslund Charlotte, 2018. "Assessing Nonresponse in a Longitudinal Establishment Survey Using Regression Trees," Journal of Official Statistics, Sciendo, vol. 34(2), pages 463-481, June.
    17. Barbara Felderer & Jannis Kueck & Martin Spindler, 2021. "Big Data meets Causal Survey Research: Understanding Nonresponse in the Recruitment of a Mixed-mode Online Panel," Papers 2102.08994, arXiv.org.
    18. Hanly, Mark & Clarke, Paul & Steele, Fiona, 2016. "Sequence analysis of call record data: exploring the role of different cost settings," LSE Research Online Documents on Economics 64896, London School of Economics and Political Science, LSE Library.
    19. Francisco Perales & Bernard Baffour & Francis Mitrou, 2015. "Ethnic Differences in the Quality of the Interview Process and Implications for Survey Analysis: The Case of Indigenous Australians," PLOS ONE, Public Library of Science, vol. 10(6), pages 1-20, June.
    20. Youjin Lee & Trang Q. Nguyen & Elizabeth A. Stuart, 2021. "Partially pooled propensity score models for average treatment effect estimation with multilevel data," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 184(4), pages 1578-1598, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:vrs:offsta:v:33:y:2017:i:3:p:753-779:n:9. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Peter Golla (email available below). General contact details of provider: https://www.sciendo.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.