Security Risks of Machine Learning Systems and Taxonomy Based on the Failure Mode Approach

This paper clarifies the source of difficulties in machine learning security and determines the usefulness of the failure mode approach for capturing security risks of machine learning systems comprehensively. Machine learning is an inductive methodology that automatically extracts relationships among data from a huge number of input-output samples. Recently, machine learning systems have been implemented deeply in various IT systems and their social impact has been increasing. However, machine learning models have specific vulnerabilities and relevant security risks that conventional IT systems do not have. An overall picture regarding these vulnerabilities and risks has not been clarified sufficiently, and there has been no consensus about their taxonomy. Thus, this paper reveals the specificity of the security risks and describes their failure modes hierarchically by classifying them on three axes, i.e., (1) presence or absence of attacker's intention, (2) location of the vulnerabilities, and (3) functional characteristics to be lost. This paper also considers points for future utilization of machine learning in society.