How to test AI: A case study of a machine learning-based trading system

The rapid evolution of artificial intelligence (AI) and especially machine learning (ML) has significantly transformed the technological landscape, influencing diverse sectors and making notable inroads into the realm of finance. This paper delves into the challenges posed by ML models, known for their black-box nature, non-deterministic behaviour, reliance on big data and inherent complexity, often lacking clear specifications. These characteristics present novel testing challenges compared to traditional software. In addressing these challenges, the authors introduce a conceptual framework tailored for testing ML-based systems, with a specific focus on financial applications. Theoretical concepts are exemplified through a real-world case study of the implementation of an ML-based bond trading system within a banking context. As the AI technologies become increasingly integrated into critical financial systems, a deeper understanding of their testing strategies will be essential for mitigating risks and harnessing their full potential. The heightened significance of in-depth testing knowledge, propelled by AI-driven progress, holds relevance for both testers and managers in navigating the complexities of the technological changes.