Multi-Agent Decisional System Based on Emotion Recognition in Speech for Controlling Exceptional Situations in Robotic Production Processes

The paper proposes a multi-agent decision-making system designed to manage exceptional situations in robotic production processes by recognizing human emotions in speech. The importance of human-robot collaboration was highlighted, with a focus on automatically detecting human operators’ emotions as potential warning signals for technical or safety issues. This approach contributes to improving the safety and efficiency of the production process by enabling agents to quickly adjust system settings according to the emotions expressed by operators in speech, for example, in cases of stress or frustration. The paper details the structural and functional scheme of the system. It presents the mathematical model and shows how deep learning algorithms implemented through hybrid neural networks (CNN and RNN) enable the system to accurately recognize and classify emotional patterns in an audio signal. The technology uses Python libraries such as TensorFlow and Keras to analyze acoustic characteristics such as tone, intensity, rhythm, and timbre of the voice. These parameters are used to identify emotions and initiate proactive or automatic corrective actions in robotic processes. Thus, the multi-agent system provides an excellent level of human-robot interaction, optimizing both productivity and safety in complex and dynamic industrial environments.