Multi-source information fusion to assess control room operator performance

Control room operators respond to abnormal situations through a series of cognitively demanding activities, e.g., monitoring, detection, diagnosis, and response. However, variability among operators in terms of prior experience and current operational context affects their response to the malfunction. A machine learning framework was employed to integrate multiple data sources and develop an empirical model of operator performance in responding to malfunction events. A human-in-the-loop within-subjects experiment was performed using a high-fidelity Generic Pressurized Water Reactor simulator. The study recruited nine licensed operators in three-person crews completing ten scenarios, each incorporating two to four malfunction events. Individual operator performance was assessed using eye tracking technology and physiological recordings of skin conductance response and respiratory function. Expert-rated event management performance was the primary study outcome. These heterogeneous data sources were fused using an approach that integrated a support vector machine with bootstrap aggregation to develop a trained quantitative prediction model. While no single variable predicted operator performance, the fused model’s predictions using independent verification data was very good (prediction accuracy of 75–83%). The proposed methodology offers a quantitative approach to evaluate the crew performance through fusing the heterogeneous data collected from experiment.