The Influence of Data Collection Method on Structural Models

In survey data, four potential sources of measurement error can jeopardize the results: the respondents, the interviewers, the questions, and the data collection method. In the past two decades, a shift has occurred in the way survey data are collected; telephone surveys and, to a lesser degree, mail surveys are now more extensively used. This has stimulated empirical research on the influence of the data collection method on data quality. Most of these mode comparisons used univariate criteria. In this study we concentrate on the potential influence of the data collection method on two substantive structural models. A controlled field experiment was conducted in which a mail, a telephone, and a face-to-face survey were compared. Using multigroup comparisons, two substantive structural equation models (one for loneliness and one for general well-being) were compared across the different data collection methods. The different data collection methods turned out to produce significantly different covariance matrices. Subsequent analyses showed that structural models, based on these covariance matrices, also differed.