Empirical study on fine-tuning pre-trained large language models for fault diagnosis of complex systems

With increasing complexity and interconnectivity of the modern industrial systems, effectively diagnosing faults is a core step to Prognostic and Health Management (PHM). Language models, particularly Large Language Models (LLMs) that pre-trained on massive corpora, have demonstrated remarkable capabilities in Natural Language Processing (NLP) and its related downstream tasks. However, how to leverage these models for facilitating system fault diagnosis and verify its effectiveness is rarely explored. This paper incorporates the potential comprehension ability of pre-trained LLMs, and investigates the efficacy of fine-tuning LLMs for realizing efficient system fault diagnosis. The experiments conducted in this study involve both open-source and closed-source models, and utilize a simulation and a real fault diagnosis dataset. We find that these models consistently achieve high performance across various metrics compared to the baselines. Additionally, qualitative and quantitative analysis is performed to investigate several aspects of the approach, such as the impact of dataset size, data normalization, missing values and explainability of the diagnosis, further showcasing the potential as well as limitations of the approach.