IDEAS home Printed from https://ideas.repec.org/a/spr/reihed/v60y2019i3d10.1007_s11162-018-9530-2.html
   My bibliography  Save this article

Should We Be Concerned About Nonresponse Bias in College Student Surveys? Evidence of Bias from a Validation Study

Author

Listed:
  • Trey Standish

    (SAS Institute, Inc)

  • Paul D. Umbach

    (North Carolina State University)

Abstract

This study uses college student survey data and corresponding administrative data on campus recreation facility usage, academic performance, physical education class attendance, and co-curricular participation to examine nonresponse bias in college student surveys. Within the context of the Groves (Public Opin Q 70:646–675, 2006) Alternative Cause Model, we found compelling evidence of the presence of nonresponse error observed as student characteristics related to the survey topic that also explain their response propensity. An individual’s survey response propensity has a statistically significant relationship with their actual behavior for 2 of 3 survey topics. In 11 of the 13 survey questions used to measure the survey topic behaviors, we found statistically significant differences between the respondent and nonrespondent behavioral measures. These findings hold important implications for survey researchers and those using student surveys for high-stakes accountability measures because survey summary statistics may not be generalizable to the target population.

Suggested Citation

  • Trey Standish & Paul D. Umbach, 2019. "Should We Be Concerned About Nonresponse Bias in College Student Surveys? Evidence of Bias from a Validation Study," Research in Higher Education, Springer;Association for Institutional Research, vol. 60(3), pages 338-357, May.
    • Handle: RePEc:spr:reihed:v:60:y:2019:i:3:d:10.1007_s11162-018-9530-2
    DOI: 10.1007/s11162-018-9530-2
    as

    Download full text from publisher

    File URL: http://link.springer.com/10.1007/s11162-018-9530-2
    File Function: Abstract
    Download Restriction: Access to the full text of the articles in this series is restricted.
    File URL: https://libkey.io/10.1007/s11162-018-9530-2?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Katharine G. Abraham & Sara E. Helms & Stanley Presser, 2008. "How Social Processes Distort Measurement: The Impact of Survey Nonresponse on Estimates of Volunteer Work," NBER Working Papers 14076, National Bureau of Economic Research, Inc.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Borgschulte, Mark & Cho, Heepyung & Lubotsky, Darren, 2022. "Partisanship and survey refusal," Journal of Economic Behavior & Organization, Elsevier, vol. 200(C), pages 332-357.
    2. Almoayad,Safa Ali Qassim & Favari,Eliana & Halabi,Samira & Krishnaswamy,Siddharth & Music,Almedina & Tandon,Sharad Alan, 2020. "Active Conflict and Access to Education : Evidence from a Series of Conflict-Related Shocks in the Republic of Yemen," Policy Research Working Paper Series 9330, The World Bank.
    3. Milton Mayfield & Jacqueline Mayfield, 2021. "Sound and Safe: The Role of Leader Motivating Language and Follower Self-Leadership in Feelings of Psychological Safety," Administrative Sciences, MDPI, vol. 11(2), pages 1-30, May.
    4. Atamanov, Aziz & Tandon, Sharad & Lopez-Acevedo, Gladys & Vergara Bahena, Mexico Alberto, 2020. "Measuring Monetary Poverty in the Middle East and North Africa (MENA) Region: Data Gaps and Different Options to Address Them," IZA Discussion Papers 13363, Institute of Labor Economics (IZA).
    5. Ann Meier & Kelly Musick & Sarah Flood & Rachel Dunifon, 2016. "Mothering Experiences: How Single Parenthood and Employment Structure the Emotional Valence of Parenting," Demography, Springer;Population Association of America (PAA), vol. 53(3), pages 649-674, June.
    6. Tiago Freire & Xiaoye Li, 2013. "How Immigration Reduced Social Capital in the US: 2005-2011," ERSA conference papers ersa13p1285, European Regional Science Association.
    7. Griffin, John & Nickerson, David & Wozniak, Abigail, 2012. "Racial differences in inequality aversion: Evidence from real world respondents in the ultimatum game," Journal of Economic Behavior & Organization, Elsevier, vol. 84(2), pages 600-617.
    8. Brick J. Michael & Tourangeau Roger, 2017. "Responsive Survey Designs for Reducing Nonresponse Bias," Journal of Official Statistics, Sciendo, vol. 33(3), pages 735-752, September.
    9. Ori Heffetz & Matthew Rabin, 2013. "Conclusions Regarding Cross-Group Differences in Happiness Depend on Difficulty of Reaching Respondents," American Economic Review, American Economic Association, vol. 103(7), pages 3001-3021, December.
    10. Hamrick, Karen S., 2012. "Nonresponse Bias Analysis of Body Mass Index in the Eating and Health Module," Technical Bulletins 184303, United States Department of Agriculture, Economic Research Service.
    11. Urszula Tymoszuk & Neta Spiro & Rosie Perkins & Adele Mason-Bertrand & Kate Gee & Aaron Williamon, 2021. "Arts engagement trends in the United Kingdom and their mental and social wellbeing implications: HEartS Survey," PLOS ONE, Public Library of Science, vol. 16(3), pages 1-35, March.
    12. Hans Schmeets & Floris Peters, 2021. "The Impact of Social Capital on Organ Donation: Evidence from the Netherlands," Social Indicators Research: An International and Interdisciplinary Journal for Quality-of-Life Measurement, Springer, vol. 157(3), pages 863-897, October.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:spr:reihed:v:60:y:2019:i:3:d:10.1007_s11162-018-9530-2. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Sonal Shukla or Springer Nature Abstracting and Indexing (email available below). General contact details of provider: http://www.springer.com .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.