Human operator decision support for highly transient industrial processes: a reinforcement learning approach

Most industrial processes are not fully-automated. Although fast and low-level control can be handled by controllers, initializing and adjusting the reference, or setpoint, values, are commonly tasks assigned to human operators. A major challenge is the control policy variation among operators. In turn this can result in inconsistencies in the final product. In order to guide operators to pursue better and more consistent performance, researchers explore the optimal control policy through different approaches. Although in different applications, researchers use different approaches, an accurate process model is still crucial to the approaches. However, for a highly transient process (e.g., the startup of a manufacturing process), modeling can be challenging and inaccurate, and approaches highly relying on a process model may not work well. In this paper, we apply the idea of offline reinforcement learning (RL), which requires the RL agent to learn control policies from a previously collected dataset. More specifically, a modified advantage weighted regression is used to guide the agent to take the more advantageous actions. In addition, we train and verify the agent by using casting data of multiple human operators from an industrial twin-roll steel strip casting process.