IDEAS home Printed from https://ideas.repec.org/p/iza/izadps/dp17347.html
   My bibliography  Save this paper

Feeling Observed? A Field Experiment on the Effects of Intense Survey Participation on Job Seekers' Labour Market Outcomes

Author

Listed:
  • Stephan, Gesine

    (Institute for Employment Research (IAB), Nuremberg)

  • Hetschko, Clemens

    (University of Leeds)

  • Schmidtke, Julia

    (Institute for Employment Research (IAB), Nuremberg)

  • Eid, Michael

    (Freie Universität Berlin)

  • Lawes, Mario

    (Freie Universität Berlin)

Abstract

We ran a field experiment to causally identify the effects of intense survey participation on key labour market outcomes. We randomly excluded individuals willing to sign up for the German Job Search Panel, a high-frequency survey with a focus on job search and well-being. Using administrative data on labour market outcomes (e.g., employment, earnings), we find that, on average, survey participation had no effect on labour market outcomes during the year after signing up. Furthermore, there is no strong heterogeneity across subgroups. Overall, this is good news for the validity of survey-based research involving labour market outcomes. We also demonstrate that a comparison of individuals signing up for the survey with individuals not responding to the invitation could have been misleading. Even when controlling for a wide range of observable characteristics, survey participation and the subsequent take up of training programs correlate significantly. This speaks to the importance of experimental research designs in our context.

Suggested Citation

  • Stephan, Gesine & Hetschko, Clemens & Schmidtke, Julia & Eid, Michael & Lawes, Mario, 2024. "Feeling Observed? A Field Experiment on the Effects of Intense Survey Participation on Job Seekers' Labour Market Outcomes," IZA Discussion Papers 17347, Institute of Labor Economics (IZA).
    • Handle: RePEc:iza:izadps:dp17347
    as

    Download full text from publisher

    File URL: https://docs.iza.org/dp17347.pdf
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Van Landeghem, Bert, 2014. "A test based on panel refreshments for panel conditioning in stated utility measures," Economics Letters, Elsevier, vol. 124(2), pages 236-238.
    2. Reichert, Arndt R. & Tauchmann, Harald, 2017. "Workforce reduction, subjective job insecurity, and mental health," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 187-212.
    3. Thomas F. Crossley & Jochem Bresser & Liam Delaney & Joachim Winter, 2017. "Can Survey Participation Alter Household Saving Behaviour?," Economic Journal, Royal Economic Society, vol. 127(606), pages 2332-2357, November.
    4. Alois Stutzer & Rafael Lalive, 2004. "The Role of Social Work Norms in Job Searching and Subjective Well-Being," Journal of the European Economic Association, MIT Press, vol. 2(4), pages 696-719, June.
    5. Kamila Cygan‐Rehm & Daniel Kuehnle & Michael Oberfichtner, 2017. "Bounding the causal effect of unemployment on mental health: Nonparametric evidence from four countries," Health Economics, John Wiley & Sons, Ltd., vol. 26(12), pages 1844-1861, December.
    6. Sonja C. Kassenboehmer & John P. Haisken‐DeNew, 2009. "You're Fired! The Causal Negative Effect of Entry Unemployment on Life Satisfaction," Economic Journal, Royal Economic Society, vol. 119(536), pages 448-462, March.
    7. Andrew E. Clark & Ed Diener & Yannis Georgellis & Richard E. Lucas, 2008. "Lags And Leads in Life Satisfaction: a Test of the Baseline Hypothesis," Economic Journal, Royal Economic Society, vol. 118(529), pages 222-243, June.
    8. Stephan, Gesine, 2016. "Arbeitsuchend, aber (noch) nicht arbeitslos: Was kommt nach der Meldung?," WSI-Mitteilungen, Nomos Verlagsgesellschaft mbH & Co. KG, vol. 69(4), pages 292-299.
    9. Marcel Das & Vera Toepoel & Arthur van Soest, 2011. "Nonparametric Tests of Panel Conditioning and Attrition Bias in Panel Surveys," Sociological Methods & Research, , vol. 40(1), pages 32-56, February.
    10. Xiaojing Dong & Ramkumar Janakiraman & Ying Xie, 2014. "The Effect of Survey Participation on Consumer Behavior: The Moderating Role of Marketing Communication," Marketing Science, INFORMS, vol. 33(4), pages 567-585, July.
    11. Ruben L. Bach & Stephanie Eckman, 2019. "Participating in a panel survey changes respondents’ labour market behaviour," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 182(1), pages 263-281, January.
    12. Thomas F. Crossley & Jochem Bresser & Liam Delaney & Joachim Winter, 2017. "Can Survey Participation Alter Household Saving Behaviour?," Economic Journal, Royal Economic Society, vol. 127(606), pages 2332-2357, November.
    13. Joseph P. Romano & Michael Wolf, 2005. "Exact and Approximate Stepdown Methods for Multiple Hypothesis Testing," Journal of the American Statistical Association, American Statistical Association, vol. 100, pages 94-108, March.
    14. Persson, Mikael, 2014. "Does Survey Participation Increase Voter Turnout? Re-examining the Hawthorne Effect in the Swedish National Election Studies," Political Science Research and Methods, Cambridge University Press, vol. 2(2), pages 297-307, October.
    15. Margherita Borella & Mariacristina De Nardi & Fang Yang, 2023. "Are Marriage-Related Taxes and Social Security Benefits Holding Back Female Labour Supply?," The Review of Economic Studies, Review of Economic Studies Ltd, vol. 90(1), pages 102-131.
    16. Steven D. Levitt & John A. List, 2011. "Was There Really a Hawthorne Effect at the Hawthorne Plant? An Analysis of the Original Illumination Experiments," American Economic Journal: Applied Economics, American Economic Association, vol. 3(1), pages 224-238, January.
    17. Johannes F. Schmieder & Till von Wachter & Jörg Heining, 2023. "The Costs of Job Displacement over the Business Cycle and Its Sources: Evidence from Germany," American Economic Review, American Economic Association, vol. 113(5), pages 1208-1254, May.
    18. Hainmueller, Jens & Xu, Yiqing, 2013. "ebalance: A Stata Package for Entropy Balancing," Journal of Statistical Software, Foundation for Open Access Statistics, vol. 54(i07).
    19. Romano, Joseph P. & Wolf, Michael, 2016. "Efficient computation of adjusted p-values for resampling-based stepdown multiple testing," Statistics & Probability Letters, Elsevier, vol. 113(C), pages 38-40.
    20. Sianesi, Barbara, 2008. "Differential effects of active labour market programs for the unemployed," Labour Economics, Elsevier, vol. 15(3), pages 370-399, June.
    21. Chadi, Adrian, 2013. "The role of interviewer encounters in panel responses on life satisfaction," Economics Letters, Elsevier, vol. 121(3), pages 550-554.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Chadi, Adrian & Hetschko, Clemens, 2025. "Income or leisure? On the hidden benefits of (un)employment," European Economic Review, Elsevier, vol. 171(C).
    2. Adrian Chadi & Clemens Hetschko, 2021. "How Job Changes Affect People's Lives — Evidence from Subjective Well‐Being Data," British Journal of Industrial Relations, London School of Economics, vol. 59(2), pages 279-306, June.
    3. Van Landeghem, Bert, 2019. "Stable traits but unstable measures? Identifying panel effects in self-reflective survey questions," Journal of Economic Psychology, Elsevier, vol. 72(C), pages 83-95.
    4. Clemens Hetschko & Andreas Knabe & Ronnie Schöb, 2019. "Looking Back in Anger? Retirement and Unemployment Scarring," Demography, Springer;Population Association of America (PAA), vol. 56(3), pages 1105-1129, June.
    5. Hetschko, Clemens & Schöb, Ronnie & Wolf, Tobias, 2020. "Income support, employment transitions and well-being," Labour Economics, Elsevier, vol. 66(C).
    6. Everding, Jakob & Marcus, Jan, 2020. "The effect of unemployment on the smoking behavior of couples," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 29(2), pages 154-170.
    7. Adrian Chadi, 2019. "Dissatisfied with life or with being interviewed? Happiness and the motivation to participate in a survey," Social Choice and Welfare, Springer;The Society for Social Choice and Welfare, vol. 53(3), pages 519-553, October.
    8. Krekel, Christian & De Neve, Jan-Emmanuel & Fancourt, Daisy & Layard, Richard, 2021. "A local community course that raises wellbeing and pro-sociality: Evidence from a randomised controlled trial," Journal of Economic Behavior & Organization, Elsevier, vol. 188(C), pages 322-336.
    9. Ubaldi, Michele & Picchio, Matteo, 2023. "Intergenerational scars: The impact of parental unemployment on individual health later in life," GLO Discussion Paper Series 1271, Global Labor Organization (GLO).
    10. Adrian Chadi & Clemens Hetschko, 2018. "The magic of the new: How job changes affect job satisfaction," Journal of Economics & Management Strategy, Wiley Blackwell, vol. 27(1), pages 23-39, March.
    11. Gábor Hajdu & Tamás Hajdu, 2024. "Does the unemployment rate moderate the well‐being disadvantage of the unemployed? Within‐region estimates from the European Social Survey," Kyklos, Wiley Blackwell, vol. 77(1), pages 40-56, February.
    12. Bert Van Landeghem & Anneleen Vandeplas, 2016. "Lower in rank, but happier: the complex relationship between status and happiness," Working Papers of LICOS - Centre for Institutions and Economic Performance 556194, KU Leuven, Faculty of Economics and Business (FEB), LICOS - Centre for Institutions and Economic Performance.
    13. Preuss, Malte & Hennecke, Juliane, 2017. "Biased by success and failure: How unemployment shapes stated locus of control," Discussion Papers 2017/29, Free University Berlin, School of Business & Economics.
    14. Botezat, Alina, 2017. "Austerity plan announcements and the impact on the employees’ wellbeing," Journal of Economic Psychology, Elsevier, vol. 63(C), pages 1-16.
    15. Adrian Chadi & Laszlo Goerke, 2023. "Seeking shelter in times of crisis? unemployment, perceived job insecurity and trade union membership," Economica, London School of Economics and Political Science, vol. 90(359), pages 1041-1088, July.
    16. Kunze, Lars & Suppa, Nicolai, 2017. "Bowling alone or bowling at all? The effect of unemployment on social participation," Journal of Economic Behavior & Organization, Elsevier, vol. 133(C), pages 213-235.
    17. Pohlan, Laura, 2019. "Unemployment and social exclusion," Journal of Economic Behavior & Organization, Elsevier, vol. 164(C), pages 273-299.
    18. Preuss, Malte & Hennecke, Juliane, 2018. "Biased by success and failure: How unemployment shapes locus of control," Labour Economics, Elsevier, vol. 53(C), pages 63-74.
    19. Chadi, Adrian, 2021. "Identification of attrition bias using different types of panel refreshments," Economics Letters, Elsevier, vol. 201(C).
    20. Clemens Hetschko, 2016. "On the misery of losing self-employment," Small Business Economics, Springer, vol. 47(2), pages 461-478, August.

    More about this item

    Keywords

    Hawthorne effect; panel conditioning; job search; labour market outcomes; field experiment;
    All these keywords.

    JEL classification:

    • C83 - Mathematical and Quantitative Methods - - Data Collection and Data Estimation Methodology; Computer Programs - - - Survey Methods; Sampling Methods
    • C93 - Mathematical and Quantitative Methods - - Design of Experiments - - - Field Experiments
    • J63 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - Turnover; Vacancies; Layoffs
    • J64 - Labor and Demographic Economics - - Mobility, Unemployment, Vacancies, and Immigrant Workers - - - Unemployment: Models, Duration, Incidence, and Job Search

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:iza:izadps:dp17347. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Holger Hinte (email available below). General contact details of provider: https://edirc.repec.org/data/izaaade.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.