A constraint partially observable semi-Markov decision process for the attack–defence relationships in various critical infrastructures

Gaining a better understanding of the relationship between attackers and defenders in cybersecurity domains to protect computer systems is of great importance. From the defender’s side, it is critical to choose the best reaction to maintain the system in a safe state, based on a given estimate of the attacker. One of the main challenges is that the defender may not be able to correctly detect a current attack due to incomplete and noisy information . Another important factor in the attack-defence interaction is the limited budget of both attackers and defenders. . This paper focuses on an approach based on interactions between the attacker and defender by considering the problem of uncertainty and limitation of resources for the defender, given that the attacker’s actions are given in all states of a Markov chain. The best actions by the defender can be characterised by a Markov Decision Process in the case of partial observability and importance of time in the expected reward, which is a Partially Observable Semi-Markov Decision model. Our simulation on a trace-based data set demonstrates that the proposed approach handles analysing interactions of the attacker and defender with limited budgets foralong with imperfect information for the defender.