IDEAS home Printed from https://ideas.repec.org/p/diw/diwwpp/dp488.html
   My bibliography  Save this paper

Validation of Survey Data on Income and Employment: The ISMIE Experience

Author

Listed:
  • Annette Jäckle
  • Emanuela Sala
  • Stephen P. Jenkins
  • Peter Lynn

Abstract

This report derives from the project "Improving survey measurement of income and employment (ISMIE)" which investigates measurement error in survey data on income and employment, using a UK sub-sample of the European Household Community Panel (ECHP). In this paper we describe the process of collecting validation data and the outcomes of the process. Validation data were obtained from two sources: employers' records and government benefit data from the Department for Work and Pensions (DWP). The former provided information on occupation and employment status, gross and net pay, membership of company pension schemes and industry sector. The latter provided histories of benefit receipt and tax credits, for example, child, disability, housing and unemployment benefits, pensions and income support. In the survey interview, respondents were asked for written permission both to obtain their DWP records and to contact their employer. They were also asked to provide information that would facilitate the process of obtaining the validation data: National Insurance number (NINO) and employer contact details. Subsequently, DWP records were extracted using a non-hierarchical matching strategy, based on different combinations of identifying variables obtained in the survey (NINO, sex, date of birth, name and postcode), and a survey of employers was carried out (mail, with telephone follow-up). The representativeness of the validation samples obtained depends on the co-operation of both survey respondents and providers of validation data, as well as errors in the matching process. We report permission rates, proportions providing matching items, match rates for the DWP data and response rates to the employer survey. We identify correlates of these measures of success at each stage of the validation process in terms of substantive characteristics of the survey respondents. Variation by subgroups is identified and implications for the representativeness of the validation sample are discussed.

Suggested Citation

  • Annette Jäckle & Emanuela Sala & Stephen P. Jenkins & Peter Lynn, 2005. "Validation of Survey Data on Income and Employment: The ISMIE Experience," Discussion Papers of DIW Berlin 488, DIW Berlin, German Institute for Economic Research.
    • Handle: RePEc:diw:diwwpp:dp488
    as

    Download full text from publisher

    File URL: https://www.diw.de/documents/publikationen/73/diw_01.c.43284.de/dp488.pdf
    Download Restriction: no
    ---><---

    Other versions of this item:

    References listed on IDEAS

    as
    1. Lynn, Peter & Sala, Emanuela, 2004. "The contact and response process in business surveys: lessons from a multimode survey of employers in the UK," ISER Working Paper Series 2004-12, Institute for Social and Economic Research.
    2. Bollinger, Christopher R, 1998. "Measurement Error in the Current Population Survey: A Nonparametric Look," Journal of Labor Economics, University of Chicago Press, vol. 16(3), pages 576-594, July.
    3. Mellow, Wesley & Sider, Hal, 1983. "Accuracy of Response in Labor Market Surveys: Evidence and Implications," Journal of Labor Economics, University of Chicago Press, vol. 1(4), pages 331-344, October.
    4. Mathiowetz, Nancy A & Duncan, Greg J, 1988. "Out of Work, Out of Mind: Response Errors in Retrospective Reports of Unemployment," Journal of Business & Economic Statistics, American Statistical Association, vol. 6(2), pages 221-229, April.
    5. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "The effects of dependent interviewing on responses to questions on income sources," ISER Working Paper Series 2004-16, Institute for Social and Economic Research.
    6. Duncan, Greg J & Hill, Daniel H, 1985. "An Investigation of the Extent and Consequences of Measurement Error in Labor-Economic Survey Data," Journal of Labor Economics, University of Chicago Press, vol. 3(4), pages 508-532, October.
    7. Bound, John & Brown, Charles & Mathiowetz, Nancy, 2001. "Measurement error in survey data," Handbook of Econometrics, in: J.J. Heckman & E.E. Leamer (ed.), Handbook of Econometrics, edition 1, volume 5, chapter 59, pages 3705-3843, Elsevier.
    8. Bound, John & Krueger, Alan B, 1991. "The Extent of Measurement Error in Longitudinal Earnings Data: Do Two Wrongs Make a Right?," Journal of Labor Economics, University of Chicago Press, vol. 9(1), pages 1-24, January.
    9. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "Linking household survey and administrative record data: what should the matching variables be?," ISER Working Paper Series 2004-23, Institute for Social and Economic Research.
    10. Stephen P. Jenkins, 2000. "Modelling household income dynamics," Journal of Population Economics, Springer;European Society for Population Economics, vol. 13(4), pages 529-567.
    11. Bound, John & Brown, Charles & Duncan, Greg J & Rodgers, Willard L, 1994. "Evidence on the Validity of Cross-Sectional and Longitudinal Labor Market Data," Journal of Labor Economics, University of Chicago Press, vol. 12(3), pages 345-368, July.
    12. Greenberg, David & Halsey, Harlan, 1983. "Systematic Misreporting and Effects of Income Maintenance Experiments on Work Effort: Evidence from the Seattle-Denver Experiment," Journal of Labor Economics, University of Chicago Press, vol. 1(4), pages 380-407, October.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Lynn, Peter & Sala, Emanuela, 2004. "The contact and response process in business surveys: lessons from a multimode survey of employers in the UK," ISER Working Paper Series 2004-12, Institute for Social and Economic Research.
    2. Stephen P. Jenkins & Lorenzo Cappellari & Peter Lynn & Annette Jäckle & Emanuela Sala, 2006. "Patterns of consent: evidence from a general household survey," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 169(4), pages 701-722, October.
    3. Peter Lynn & Annette Jäckle & Stephen P. Jenkins & Emanuela Sala, 2012. "The impact of questioning method on measurement error in panel survey measures of benefit receipt: evidence from a validation study," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 175(1), pages 289-308, January.
    4. Ermisch, John & Gambetta, Diego, 2006. "People’s trust: the design of a survey-based experiment," ISER Working Paper Series 2006-34, Institute for Social and Economic Research.
    5. John Ermisch & Diego Gambetta & Heather Laurie & Thomas Siedler & S. C. Noah Uhrig, 2009. "Measuring people's trust," Journal of the Royal Statistical Society Series A, Royal Statistical Society, vol. 172(4), pages 749-769, October.
    6. Lynn, Peter & Sala, Emanuela, 2005. "The impact of a mixed-mode data collection design on non response bias on a business survey," ISER Working Paper Series 2005-16, Institute for Social and Economic Research.
    7. Ermisch, John & Gambetta, Diego, 2010. "Do strong family ties inhibit trust?," Journal of Economic Behavior & Organization, Elsevier, vol. 75(3), pages 365-376, September.
    8. Lynn, Peter & Sala, Emanuela & Noah Uhrig, S.C., 2008. "The development and implementation of a coding scheme to analyse interview dynamics in the British Household Panel Survey," ISER Working Paper Series 2008-19, Institute for Social and Economic Research.
    9. Lugtig Peter & Jäckle Annette, 2014. "Can I Just Check...? Effects of Edit Check Questions on Measurement Error and Survey Estimates," Journal of Official Statistics, Sciendo, vol. 30(1), pages 45-62, March.
    10. Alexia Meyermann & Jennifer Elsner & Jürgen Schupp & Stefan Liebig, 2009. "Pilotstudie einer surveybasierten Verknüpfung von Personen- und Betriebsdaten: Durchführung sowie Generierung einer Betriebsstudie als nachgelagerte Organisationserhebung zur SOEP-Innovationsstichprob," SOEPpapers on Multidisciplinary Panel Data Research 170, DIW Berlin, The German Socio-Economic Panel (SOEP).
    11. Emanuela Sala & Peter Lynn, 2009. "The potential of a multi-mode data collection design to reduce non response bias. The case of a survey of employers," Quality & Quantity: International Journal of Methodology, Springer, vol. 43(1), pages 123-136, January.
    12. Jäckle, Annette, 2006. "Dependent interviewing: a framework and application to current research," ISER Working Paper Series 2006-32, Institute for Social and Economic Research.
    13. Ermisch, John & Gambetta, Diego, 2011. "The Long Shadow of Income on Trustworthiness," IZA Discussion Papers 5585, Institute of Labor Economics (IZA).
    14. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "Linking household survey and administrative record data: what should the matching variables be?," ISER Working Paper Series 2004-23, Institute for Social and Economic Research.
    15. Lynn, Peter & Sala, Emanuela, 2004. "Measuring change in employment characteristics: the effects of dependent interviewing," ISER Working Paper Series 2004-26, Institute for Social and Economic Research.
    16. Korbmacher, Julie M. & Schröder, Mathis, 2013. "Consent when Linking Survey Data with Administrative Records: The Role of the Interviewer," EconStor Open Access Articles and Book Chapters, ZBW - Leibniz Information Centre for Economics, vol. 7(2), pages 115-131.
    17. SC Noah Uhrig & Emanuela Sala, 2011. "When Change Matters: An Analysis of Survey Interaction in Dependent Interviewing on the British Household Panel Study," Sociological Methods & Research, , vol. 40(2), pages 333-366, May.
    18. Lynn, Peter & Jäckle, Annette & Sala, Emanuela & P. Jenkins, Stephen, 2004. "The impact of interviewing method on measurement error in panel survey measures of benefit receipt: evidence from a validation study," ISER Working Paper Series 2004-28, Institute for Social and Economic Research.
    19. Jäckle, Annette & Lugtig, Peter, 2011. "Can I just check…? Effects of edit check questions on measurement error and survey estimates," ISER Working Paper Series 2011-23, Institute for Social and Economic Research.
    20. Lynn, Peter & Jäckle, Annette, 2004. "Dependent interviewing and seam effects in work history data," ISER Working Paper Series 2004-24, Institute for Social and Economic Research.
    21. Sala, Emanuela & Noah Uhrig, S.C., 2009. "When change matters: the effect of dependent interviewing on survey interaction in the British Household Panel Study," ISER Working Paper Series 2009-09, Institute for Social and Economic Research.
    22. Jäckle, Annette, 2005. "Does dependent interviewing really increase efficiency and reduce respondent burden?," ISER Working Paper Series 2005-11, Institute for Social and Economic Research.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Bollinger, Christopher R. & Hirsch, Barry & Hokayem, Charles M. & Ziliak, James P., 2018. "Trouble in the Tails? What We Know about Earnings Nonresponse Thirty Years after Lillard, Smith, and Welch," IZA Discussion Papers 11710, Institute of Labor Economics (IZA).
    2. Jungmin Lee & Sokbae Lee, 2012. "Does it Matter WHO Responded to the Survey? Trends in the U.S. Gender Earnings Gap Revisited," ILR Review, Cornell University, ILR School, vol. 65(1), pages 148-160, January.
    3. Kristensen, Nicolai & Westergård-Nielsen, Niels C., 2006. "A Large-Scale Validation Study of Measurement Errors in Longitudinal Survey Data," IZA Discussion Papers 2329, Institute of Labor Economics (IZA).
    4. John Abowd & Martha Stinson, 2011. "Estimating Measurement Error in SIPP Annual Job Earnings: A Comparison of Census Bureau Survey and SSA Administrative Data," Working Papers 11-20, Center for Economic Studies, U.S. Census Bureau.
    5. Peter Gottschalk & Minh Huynh, 2010. "Are Earnings Inequality and Mobility Overstated? The Impact of Nonclassical Measurement Error," The Review of Economics and Statistics, MIT Press, vol. 92(2), pages 302-315, May.
    6. de Nicola, Francesca & Giné, Xavier, 2014. "How accurate are recall data? Evidence from coastal India," Journal of Development Economics, Elsevier, vol. 106(C), pages 52-65.
    7. Brownstone, David & Valletta, Robert G, 1996. "Modeling Earnings Measurement Error: A Multiple Imputation Approach," The Review of Economics and Statistics, MIT Press, vol. 78(4), pages 705-717, November.
    8. Arie Kapteyn & Jelmer Y. Ypma, 2007. "Measurement Error and Misclassification: A Comparison of Survey and Administrative Data," Journal of Labor Economics, University of Chicago Press, vol. 25(3), pages 513-551.
    9. Burt S. Barnow & David Greenberg, 2015. "Do Estimated Impacts on Earnings Depend on the Source of the Data Used to Measure Them? Evidence From Previous Social Experiments," Evaluation Review, , vol. 39(2), pages 179-228, April.
    10. Andrew S. Green, 2017. "Hours Off the Clock," Working Papers 17-44, Center for Economic Studies, U.S. Census Bureau.
    11. ChangHwan Kim & Christopher R. Tamborini, 2014. "Response Error in Earnings," Sociological Methods & Research, , vol. 43(1), pages 39-72, February.
    12. Liu, Long, 2009. "On hourly wages and weekly earnings in the current population survey," Economics Letters, Elsevier, vol. 105(1), pages 113-116, October.
    13. Bollinger, Christopher R, 1998. "Measurement Error in the Current Population Survey: A Nonparametric Look," Journal of Labor Economics, University of Chicago Press, vol. 16(3), pages 576-594, July.
    14. Jesse Bricker & Gary V. Engelhardt, 2007. "Measurement Error in Earnings Data in the Health and Retirement Study," Working Papers, Center for Retirement Research at Boston College wp2007-16, Center for Retirement Research, revised Oct 2007.
    15. Christopher R. Bollinger, 2001. "Response Error and the Union Wage Differential," Southern Economic Journal, John Wiley & Sons, vol. 68(1), pages 60-76, July.
    16. Fisher, Jonathan D. & Houseworth, Christina A., 2013. "Occupation inflation in the Current Population Survey," Journal of Economic and Social Measurement, IOS Press, issue 3, pages 243-261.
    17. Nicolas Frémeaux, 2023. "The More, the Better? Individual and Joint Interviewing in Surveys," Annals of Economics and Statistics, GENES, issue 149, pages 63-96.
    18. Lisa M. Dragoset & Gary S. Fields, 2006. "U.S. Earnings Mobility: Comparing Survey-Based and Administrative-Based Estimates," Working Papers 55, ECINEQ, Society for the Study of Economic Inequality.
    19. Michele Lalla & Patrizio Frederic & Daniela Mantovani, 2022. "The inextricable association of measurement errors and tax evasion as examined through a microanalysis of survey data matched with fiscal data: a case study," Statistical Methods & Applications, Springer;Società Italiana di Statistica, vol. 31(5), pages 1375-1401, December.
    20. Martin H. David & Christopher R. Bollinger, 2005. "I didn't tell, and I won't tell: dynamic response error in the SIPP," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(4), pages 563-569.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:diw:diwwpp:dp488. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Bibliothek (email available below). General contact details of provider: https://edirc.repec.org/data/diwbede.html .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.