IDEAS home Printed from https://ideas.repec.org/a/dem/demres/v43y2020i50.html
   My bibliography  Save this article

Evaluating interviewer manipulation in the new round of the Generations and Gender Survey

Author

Listed:
  • Eugenio Paglino

    (Helsingin Yliopisto (University of Helsinki))

  • Tom Emery

    (Erasmus Universiteit Rotterdam)

Abstract

Background: Past research has criticized the quality of the Generations and Gender Survey retrospective fertility and partnership histories. For example, fatigue and learning effects were deemed responsible for distortions in the Generations and Gender Survey in Germany. Objective: We assess the quality of the Generations and Gender Survey for Belarus (GGS-BL) in 2017 to assess whether the new centralized fieldwork system and monitoring procedures are effective in preventing distortions in life history data. Methods: We conduct a range of analyses to find evidence of fatigue and learning effects on the part of both interviewers and respondents. Multilevel models, comparison of crucial indicators with other sources, and descriptive analysis of item-nonresponse are used. Results: In a preliminary analysis, we find no evidence of severe distortions. An in-depth analysis into interviewer and respondent effects reveals some small signs of possible manipulation. However, when assessing the impact of anomalous interviewers on the indicators more likely to be affected, we find no evidence of harm to data quality. Conclusions: The new data collection procedure adopted by the Generations and Gender Survey seems to be effective in preventing detectable manipulation and fabrication. Furthermore, we dismiss the hypothesis that fatigue and learning effects are a source of bias in the collection of life history data. Contribution: This paper delivers three key messages: (1) the Generations and Gender Survey for Belarus is a reliable source for retrospective histories, (2) in-field checks are an effective tool to prevent fabrication, and (3) extensive use of inexperienced interviewers does not seem to harm data quality when adequate monitoring and monitoring is in place.

Suggested Citation

  • Eugenio Paglino & Tom Emery, 2020. "Evaluating interviewer manipulation in the new round of the Generations and Gender Survey," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 43(50), pages 1461-1494.
    • Handle: RePEc:dem:demres:v:43:y:2020:i:50
    DOI: 10.4054/DemRes.2020.43.50
    as

    Download full text from publisher

    File URL: https://www.demographic-research.org/volumes/vol43/50/43-50.pdf
    Download Restriction: no
    File URL: https://libkey.io/10.4054/DemRes.2020.43.50?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    References listed on IDEAS

    as
    1. Su-Hao Tu & Pei-Shan Liao, 2007. "Social Distance, Respondent Cooperation and Item Nonresponse in Sex Survey," Quality & Quantity: International Journal of Methodology, Springer, vol. 41(2), pages 177-199, April.
    2. De Haas Samuel & Winker Peter, 2016. "Detecting Fraudulent Interviewers by Improved Clustering Methods – The Case of Falsifications of Answers to Parts of a Questionnaire," Journal of Official Statistics, Sciendo, vol. 32(3), pages 643-660, September.
    3. Michaela R. Kreyenfeld & Anne Hornung & Karolin Kubisch & Ina Jaschinski, 2010. "Fertility and union histories from German GGS data: some critical reflections," MPIDR Working Papers WP-2010-023, Max Planck Institute for Demographic Research, Rostock, Germany.
    4. Christin Schäfer & Jörg-Peter Schräpler & Klaus-Robert Müller & Gert G. Wagner, 2004. "Automatic Identification of Faked and Fraudulent Interviews in Surveys by Two Different Methods," Discussion Papers of DIW Berlin 441, DIW Berlin, German Institute for Economic Research.
    5. Jorik Vergauwen & David De Wachter & Karel Neels & Jonas Wood, 2015. "Quality of demographic data in GGS Wave 1," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 32(24), pages 723-774.
    6. Giles POLGLASE, 2013. "Higher education as soft power in the Eastern Partnership: the case of Belarus," Eastern Journal of European Studies, Centre for European Studies, Alexandru Ioan Cuza University, vol. 4, pages 111-121, December.
    7. Darren W. Davis & Brian D. Silver, 2003. "Stereotype Threat and Race of Interviewer Effects in a Survey on Political Knowledge," American Journal of Political Science, John Wiley & Sons, vol. 47(1), pages 33-45, January.
    8. Sebastian Klüsener & Aiva Jasilioniene & Victoriya Yuodeshko, 2019. "Retraditionalization as a pathway to escape lowest-low fertility? Characteristics and prospects of the Eastern European “baby boom”," MPIDR Working Papers WP-2019-014, Max Planck Institute for Demographic Research, Rostock, Germany.
    9. Kerstin Ruckdeschel & Lenore Sauer & Robert Naderi, 2016. "Reliability of retrospective event histories within the German Generations and Gender Survey," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 34(11), pages 321-358.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Kerstin Ruckdeschel & Lenore Sauer & Robert Naderi, 2016. "Reliability of retrospective event histories within the German Generations and Gender Survey," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 34(11), pages 321-358.
    2. Judith C. Koops & Aart C. Liefbroer & Anne H. Gauthier, 2017. "The Influence of Parental Educational Attainment on the Partnership Context at First Birth in 16 Western Societies," European Journal of Population, Springer;European Association for Population Studies, vol. 33(4), pages 533-557, October.
    3. Jenny Gierveld & Pearl A. Dykstra & Niels Schenk, 2012. "Living arrangements, intergenerational support types and older adult loneliness in Eastern and Western Europe," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 27(7), pages 167-200.
    4. Brienna Perelli-Harris & Michaela Kreyenfeld & Wendy Sigle-Rushton & Renske Keizer & Trude Lappegård & Aiva Jasilioniene & Caroline Berghammer & Paola Di Giulio, 2012. "Changes in union status during the transition to parenthood in eleven European countries, 1970s to early 2000s," Population Studies, Taylor & Francis Journals, vol. 66(2), pages 167-182, July.
    5. Tricia Koroknay†Palicz & Joao Montalvao, 2020. "Sex, Lies, and Surveys: The Role of Interviewer Characteristics," Economics Bulletin, AccessEcon, vol. 40(4), pages 3313-3324.
    6. Tineke Fokkema & Aart C. Liefbroer & Nicole Hiekel & Tom Emery & Andrej Kveder, 2016. "Generations and Gender Programme Wave 1 data collection," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 34(18), pages 499-524.
    7. Alessandra Trimarchi & Jan Van Bavel, 2017. "Pathways to marital and non-marital first birth: the role of his and her education," Vienna Yearbook of Population Research, Vienna Institute of Demography (VID) of the Austrian Academy of Sciences in Vienna, vol. 15(1), pages 143-179.
    8. Kosyakova, Yuliya & Olbrich, Lukas & Sakshaug, Joseph & Schwanhäuser, Silvia, 2019. "Identification of interviewer falsification in the IAB-BAMF-SOEP Survey of Refugees in Germany," FDZ Methodenreport 201902_en, Institut für Arbeitsmarkt- und Berufsforschung (IAB), Nürnberg [Institute for Employment Research, Nuremberg, Germany].
    9. Harris, J. Andrew & van der Windt, Peter, 2023. "Empowering women or increasing response bias? Experimental evidence from Congo," Journal of Development Economics, Elsevier, vol. 164(C).
    10. Katharina Wolf, 2014. "Fertility of Turkish migrants in Germany: duration of stay matters," MPIDR Working Papers WP-2014-001, Max Planck Institute for Demographic Research, Rostock, Germany.
    11. Brienna Perelli-Harris & Mark Amos, 2015. "Changes in partnership patterns across the life course," Demographic Research, Max Planck Institute for Demographic Research, Rostock, Germany, vol. 33(6), pages 145-178.
    12. Himelein,Kristen, 2015. "Interviewer effects in subjective survey questions: evidence from Timor-Leste," Policy Research Working Paper Series 7208, The World Bank.
    13. Bonaccolto-Töpfer, Marina & Briel, Stephanie, 2022. "The gender pay gap revisited: Does machine learning offer new insights?," Labour Economics, Elsevier, vol. 78(C).
    14. Maria Winkler-Dworak & Eva Beaujouan & Paola Di Giulio & Martin Spielauer, 2019. "Simulating Family Life Courses: An Application for Italy, Great Britain, and Scandinavia," VID Working Papers 1908, Vienna Institute of Demography (VID) of the Austrian Academy of Sciences in Vienna.
    15. Niklas Egels-Zandén, 2014. "Revisiting Supplier Compliance with MNC Codes of Conduct: Recoupling Policy and Practice at Chinese Toy Suppliers," Journal of Business Ethics, Springer, vol. 119(1), pages 59-75, January.
    16. Pan, Suyan, 2024. "Discursive imaginaries and lived realities in the making of an education hub: The case of Hong Kong," International Journal of Educational Development, Elsevier, vol. 105(C).
    17. Michaela R. Kreyenfeld & Valerie Martin, 2011. "Economic conditions of stepfamilies from a cross-national perspective," MPIDR Working Papers WP-2011-010, Max Planck Institute for Demographic Research, Rostock, Germany.
    18. Su-Hao Tu & Pei-Shan Liao, 2007. "Social Distance, Respondent Cooperation and Item Nonresponse in Sex Survey," Quality & Quantity: International Journal of Methodology, Springer, vol. 41(2), pages 177-199, April.
    19. Jonas Wood & Karel Neels & Jorik Vergauwen, 2016. "Economic and Institutional Context and Second Births in Seven European Countries," Population Research and Policy Review, Springer;Southern Demographic Association (SDA), vol. 35(3), pages 305-325, June.
    20. Olbrich, Lukas & Kosyakova, Yuliya & Sakshaug, Joseph W., 2022. "The reliability of adult self-reported height: The role of interviewers," Economics & Human Biology, Elsevier, vol. 45(C).

    More about this item

    Keywords

    survey methods; interviewer effects; Generations and Gender Survey (GGS); data quality; retrospective histories;
    All these keywords.

    JEL classification:

    • J1 - Labor and Demographic Economics - - Demographic Economics
    • Z0 - Other Special Topics - - General

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:dem:demres:v:43:y:2020:i:50. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Editorial Office (email available below). General contact details of provider: https://www.demogr.mpg.de/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.