Accounting for Individual-Specific Reliability of Self-Assessed Measures of Economic Preferences and Personality Traits

Measures based on self-assessments, which are increasingly important in empirical economic research, are plagued by measurement error. This paper presents the first attempt at measuring both revealed and self-reported reliability of individuals’ answers on self-reports of latent characteristics. We show that measurement error on self-reports relevant to economists is heterogeneous across individuals and can be reasonably approximated by a distribution with two unobserved types. We propose a straightforward survey question which allows to distinguish individuals who give highly reliable answers from those who do not, using cross-sectional data. We demonstrate that it predicts revealed individual reliability over and above all measured characteristics, survey conditions, and experimental treatments. We show how our simple self-reported reliability measure can be used to cost-effectively reduce attenuation bias in estimates of cognitive and non-cognitive determinants of high school GPA, college graduation, unemployment, and life satisfaction. Without requiring panel data, the achieved correction is similar to some of the most effective reduced-form theory-based approaches in the existing literature. Finally, we clarify the role of effort and self-knowledge in generating measurement error and propose a simple model which rationalizes our findings.