Pilot Contamination Attack Detection Methods—An Exhaustive Performance Evaluation Through Probability Metrics and Statistical Classification Parameters

Among the numerous strategies that an attacker can initiate to enhance its eavesdropping capabilities is the Pilot Contamination Attack (PCA). Two promising methods, based on Phase-Shift Keying (PSK) modulation of Nth order— 2-N-PSK and Shifted 2-N-PSK , can detect an existing PCA by means of analysis of the constellation that the correlation product of received pilot signals belongs to. The overall efficiency of the methods can be studied by the most commonly used probability metrics—detection probability and false alarm probability. However, this information may be insufficient for comparison purposes; therefore, to acquire a more holistic perspective on the methods’ performances, statistical evaluation metrics can be obtained. Depending on the particular application of the system in which the PCA detection methods are incorporated and the distribution of attack initiation among all samples, different classification parameters are of varying significance in the efficiency assessment. In this paper, 2-N-PSK and Shifted 2-N-PSK are comprehensively studied through their probability parameters. In addition, the methods are also compared by their most informative statistical parameters, such as accuracy , precision and recall , F1-score , specificity , and fall-out . A large number of simulations are carried out, the analyses of which indisputably prove the superior behavior of the Shifted 2-N-PSK compared to the 2-N-PSK detection method. Since a method’s performance is strongly related to the number of antenna elements at the base station, all simulations are conducted for scenarios with different antennae numbers. The most promising realization of Shifted 2-N-PSK improves the receiver operating characteristics results of the original 2-N-PSK by 7.38%, 4.33%, and 5.61%, and outperforms the precision recall analyses of 2-N-PSK by 10.02%, 4.82% and 3.86%, for the respective number of 10, 100 and 300 antenna elements at the base station.