A Proposed Framework for Artificial Intelligence Safety and Technology Readiness Assessments for National Security Applications

The integration of artificial intelligence (AI) within the national security sector is increasing, yet traditional Technology Readiness Level (TRL) assessments fall short in evaluating the maturity of AI-enabled systems. Challenges specific to AI, such as alignment, explainability, and control, complicate the assessment of technical risk using the TRL framework. To address this gap, we introduce an AI Readiness Level (AIRL) framework, modified from TRL, to better evaluate the risk and readiness of AI systems in national security. The AIRL framework focuses on five key criteria: alignment, justified confidence, governance, human readiness level (HRL), and data readiness level (DRL). By assessing these dimensions, AIRL identifies minimum standards for deployment, areas for improvement, and technical risks. We apply the AIRL framework to hypothetical AI systems, revealing that despite high TRL ratings, these systems often fall short in critical AI-specific areas. AIRL offers program managers guidance on advancing AI systems to higher readiness levels, with a focus on improving alignment, transparency, governance, user interfaces, and data management. The framework provides valuable insights for policymakers, developers, and operators to ensure AI solutions meet ethical and performance standards in government operations.