Evaluating Measurement Invariance of TALIS 2013 Complex Scales: Comparison between Continuous and Categorical Multiple-Group Confirmatory Factor Analyses

This paper evaluates measurement invariance of complex scales from a social survey using both a continuous approach and a categorical approach to help inform future decisions in choosing the most appropriate methods to perform the validation of complex scales. In particular, continuous and categorical approaches are compared for constructing and validating 11 complex scales across 23 countries participating in the first round of the OECD Teaching and Learning International Survey (TALIS). Two invariance testing approaches were compared – 1) continuous multiple-group confirmatory factor analysis; 2) categorical multiple-group confirmatory factor analysis. Latent variable modelling was employed to account for the complex structure of the relationships between many items in each scale. The performance of the models is reported and illustrated based on the evaluation of the level of measurement invariance. All of the scales established configural and metric levels of invariance from both approaches, and three scales established scalar invariance from the categorical approach, allowing for a meaningful mean score comparison across countries. Limitations of the models compared in this study and future considerations for construction and validation of scaling complex scales are discussed.