Tackling Verification and Validation Techniques to Evaluate Cyber Situational Awareness Capabilities

Since cyberspace was identified as a domain of operations, defence practitioners started a race with academy, researchers, and industry and military organizations working together towards defining related lines of capability development (e.g., DOTMLPFI) and exploring the needs and opportunities they entail. An essential cornerstone of adapting to the convergence of the cyber domain with conventional theaters of operation is the need for producing tools for easing to acquire cyber situational awareness (CSA), from which human operators shall be able to perceive, reason and project situations and events observed in cyberspace that may vertically/horizontally propagate from technological to tactical, operational and strategic planes. Benefiting from the higher maturity level of civilian capabilities for cybersecurity, the military sector has embraced the challenge of creating related beyond state-of-the-art CSA enablers that comprise the existing technological background while adopting concepts such as operations, missions or courses of action (CoAs), properly aligning them with military doctrine. Beyond ongoing development efforts, there is a wide methodological gap in the lack of suitable CSA verification and validation (V&V) frameworks, which are expected to analyze if related capabilities meet the requirements to operate in the military context; at the same time supporting the thorough development life-cycle of brand new cyber defence technologies. With the motivation of closing the identified gap, this research introduces a novel V&V framework able to guide the evaluation of CSA-related tools, which makes converge purely military aspects with dual-use state-of-the-art V&V approaches. Three core CSA evaluation concepts are discussed in-depth: software, operational and application tests. They range from the daily application of new capabilities to their ability to enable the acquisition of a joint operational picture understandable by human decision makers.