Conditioning: How background variables can influence PISA scores

The Programme for International Student Assessment (PISA) is an international large-scale assessment which examines the educational achievement of 15-year-old students across the world. It has long become one of the key studies for evidence-based education policymaking across the globe. As result, PISA results and the methodology that they are based on should be robust, open and transparent. Yet, PISA receives significant criticism for its scaling model and the opaqueness in communicating it. One particular point of concern is the so-called "conditioning model", where background variables are used in the derivation of student achievement scores. The aim of this paper is to investigate this part of the scaling model and the impact it has upon the final scores. This includes varying the background variables of the conditioning model systematically and analysing the impact that this has on multiple measures. Our key finding is that the exact specification of the conditioning model matters and has substantial impact on average scores in some of the minor PISA domains (namely reading). It also has a major impact upon cross-national comparisons of educational inequality.