Deploying Predictive Models for a Process-Aware Decision Support System

(a) Situation faced: In recent years, there has been increasing interest in using machine learning (ML) in industrial applications, such as loan approval process and travel approval process to support decision-making in business processes. This case involved building an ML-based decision support system within the IBM workflow engine that can be used to help knowledge workers make high-quality decisions while executing a business process. The deployment of ML in such systems poses various challenges and concerns. (b) Action taken: There are several requirements for the production of ML systems and significant differences between evaluations in the academic setting and what is required of a real-world system. This includes the ability to cope with dynamic and changing data, model retraining to prevent model decay, low computational costs, and accurate and interpretable models that promote user trust. We present the design considerations for building a process-aware decision support system for business processes and outline the operational decisions of the process-aware feature engineering pipeline, how to choose a machine learning algorithm, and how to carry out end-user model testing. We also present our approach to handle model decay via online training that incorporates user feedback. (c) Results achieved: We successfully deployed the actions in ML-based DSS within the IBM workflow engine by addressing the requirements of various phases of the ML deployment including data preparation, model selection, model verification, and model deployment. (d) Lessons learned: The deployment of an ML-based DSS as a part of a workflow engine presents a distinct set of issues invalidating common assumptions. Despite having complex deep learning-based models increasingly developed and benchmarked by the research community, in practice, simpler models with low computational requirements are often chosen. The absence of real-world data with all the necessary characteristics for model verification often requires a synthetic process to generate data representative of the real world that includes drift. User feedback plays an important role in improving the decisions made by ML-based DSSs.