Total least squares bias in climate fingerprinting regressions with heterogeneous noise variances and correlated explanatory variables

Regression‐based “fingerprinting” methods in climate science employ total least squares (TLS) or orthogonal regression to remedy attenuation bias arising from measurement error due to reliance on climate model‐generated explanatory variables. Proving the consistency of multivariate TLS requires assuming noise variances are equal across all variables in the model. This assumption has been challenged empirically in the climate context but little is known about TLS biases when the assumption is violated. Monte Carlo analysis is used herein to examine coefficient biases when the noise variances are not equal. The analysis allows the explanatory variables to be negatively correlated which is typical in climate applications. Ordinary least squares (OLS) exhibits the expected attenuation bias which vanishes as the noise variances on the explanatory variables disappear. In some cases, TLS corrects attenuation bias but more typically imparts large and generally positive biases. OLS performs well when the true value of β=0$$ \beta =0 $$ whereas TLS performs quite poorly. This implies that TLS is not well suited for tests of the null. When β=1$$ \beta =1 $$ TLS tends to exhibit opposite biases to OLS. Diagnostic information specific to each data sample should be consulted before using TLS to avoid spurious inferences and replacing OLS attenuation bias with other, potentially larger biases.