Knowledge distillation-based domain generalization enabling invariant feature distributions for damage detection of rotating machines and structures

The poor generalization ability of machine learning (ML) models in the absence of sufficient labeled data remains a major challenge hindering their practical application. Domain generalization (DG) allows ML models trained on a set of source domains to generalize directly to related but unseen target domains. This capability makes DG particularly suitable for migrating ML model to unseen structures for online structural damage detection. A main branch of DG methods is domain-invariant feature learning, which involves aligning extracted embedding to make it invariant across domains. However, as the domain variety increases, extracting domain-invariant features becomes more challenging. This study rethinks DG from a novel three-stage knowledge distillation perspective. The first stage learns features with domain-invariant conditional distribution based on variational Bayesian inference. Multiple auxiliary domain-specific student models are designed to establish a bridge between unconditional and conditional variational inference. In the second stage, auxiliary student models learn from each other to additionally excavate features with domain-invariant marginal distribution. In the third stage, a student leader model distills knowledge from all auxiliary student models to enhance the model's robustness for final decision-making. The developed method is applied to civil and mechanical structures for damage detection. Results demonstrate that the developed method outperforms the state-of-the-art DG methods. Besides, the designed mechanism enables the student leader model to achieve superior performance compared to the teacher model.