A Distributed Machine Learning-Based Scheme for Real-Time Highway Traffic Flow Prediction in Internet of Vehicles

Abnormal traffic flow prediction is crucial for reducing traffic congestion. Most recent studies utilized machine learning models in traffic flow detection systems. However, these detection systems do not support real-time analysis. Centralized machine learning methods face a number of challenges due to the sheer volume of traffic data that needs to be processed in real-time. Thus, it is not scalable and lacks fault tolerance and data privacy. This study designs and evaluates a scalable distributed machine learning-based scheme to predict highway traffic flows in real-time. The proposed system is segment-based where the vehicles in each segment form a cluster. We train and validate a local Random Forest Regression (RFR) model for each vehicle’s cluster (highway-segment) using six different hyper parameters. Due to the variance of traffic flow patterns between segments, we build a global Distributed Machine Learning Random Forest (DMLRF) regression model to improve the system performance for abnormal traffic flows. Kappa Architecture is utilized to enable real-time prediction. The proposed model is evaluated and compared to other base-line models, Linear Regression (LR), Logistic Regression (LogR), and K Nearest Neighbor (KNN) regression in terms of Mean Square Error (MSE), Root Mean Square Error (RMSE), Mean Absolute Error (MAE), R-squared (R 2 ), and Adjusted R-Squared (AR 2 ). The proposed scheme demonstrates high accuracy in predicting abnormal traffic flows while maintaining scalability and data privacy.