Solving traffic queues at controlled-signalized intersections in continuous-time Markov games

The traffic signal control plays a fundamental role to improve the efficiency and efficacy of traffic flows in traffic networks. This paper is the first work in which we consider a mathematically rigorous study of the continuous-time, discrete state, multi-traffic signal control problem using a non-cooperative game theory approach. The solution of the problem is circumscribed to an ergodic, controllable, discrete state, continuous-time Markov game computed under the expected average cost criterion. This paper provides several main contributions. First, we present a general continuous-time queue model, which is employed as the fundamental scheme of a computationally tractable game theory approach for the signal control continuous-time Markov game. This model is transformed into a discrete state Poisson process where the vehicles leave the queue in the order they arrive. Second, in this problem, each signal controller (player) aims at finding green time that minimizes its signal and queuing delay. Then, a conflict appears when each signal controller tries to minimize its queue. We study the problem of computing a Nash equilibrium for this game. Our third contribution employs a proximal/gradient method for computing the Nash equilibrium point of the game. By introducing new restrictions over the signal controller and adding a restriction for continuous-time Markov chains, we obtain the set of average optimal policies, which is one of the main results of this paper. Hence, our final contribution shows, in simulation, the usefulness of the proposed method with an application example.