Dynamic Traffic Management of Highway Networks

Abstract

Efficient operation of traffic networks via management strategies can guarantee overall societal benefits for both the humans and the environment. As the number of vehicles and the need for transportation grows, dynamic traffic management aims to increase the safety and efficiency of the traffic networks without the need to change the infrastructure of the existing roads. Since the highway networks are considered permanent investments that are expensive to build and maintain, the main scope of this dissertation is to propose traffic flow models and methods to improve the efficiency of the current highway systems without the need to change their infrastructure. When all vehicles in a network are \textit{Human-Driven Vehicles} (HDVs), and changing the infrastructure is either so expensive or impossible, then one reasonable approach to improve the efficiency of traffic networks is through the control of traffic signal lights specially because the behavior of the human drivers cannot be directly controlled. A literature review of highway traffic control demonstrate that \textit{Ramp Metering} (RM) is one of the most commonly used approaches as it improves the network performance in regards to travel time, travel distance, throughput, etc and cost-wise, it is a very economical approach. As such, in this research, the ultimate goal focus is to extend the current literature on traffic managements of highway networks by offering new models and algorithms to improve this field. To reach this goal, the first step is to focus on improving and extending the current traffic flow models. There are two categories of traffic flow models in the literature: First-order models, and Second-order models. Many different extensions of the famous first-order model called the Cell-Transmission Model (CTM) have been proposed throughout the past decades, each one proposed based on different criteria and the specific needs of different applications. In the first part of this dissertation, a performance assessment of the most important extensions of CTM will be performed. Then, based on this evaluation, an extended version of the CTM, called the Piece-Wise Affine Approximation-CTM (\textit{PWA-CTM}), will be offered which will be proven to have better performance regarding the evolution of traffic flow and computation time comparing to the previous versions of this model. In the next step, the focus will be shifted to second-order models as they have better capabilities of modeling the behavior of traffic flow comparing to the first-order models. However, any optimization scheme for highway traffic control based on these models is highly nonlinear and computationally intensive. As such, in this part of the research, a linearization of the famous second-order model called the \textit{METANET} will be offered which is based on PWA approximations and also synthetic data generation techniques. With extensive simulations, it will be shown that this linearized approximation can greatly impact the computational complexity of any optimization-based traffic control framework based on this second-order traffic flow model. Moreover, to have significant traffic management improvements, not only the underlying traffic models, but also the control strategies should be enhanced. The availability of increasing computational power and sensing and communication capabilities, as well as advances in the field of machine learning, has developed \textit{learning-based} control approaches which can address constraint satisfaction and closed-loop performance optimization. In this chapter, \textit{Reinforcement Learning} (RL) algorithms will be investigated to solve the optimal control problem of RM. In the case of RM, RL-based techniques offer a potentially appealing alternative method to solve the problem at hand, since they are data-based and make no assumptions on the underlying model parameters. Towards this direction, it is convenient to study the road model as a multi-agent system of non-homogeneous networked agents. In the following, a novel formulation of the RM problem as an optimal control problem based on a first-order multi-agent dynamical system will be offered. Then, applying policy gradient RL algorithms, a probabilistic policy will be found that solves the ramp-metering problem. The performance of the optimal policy learnt will be investigated under different scenarios to evaluate its efficiency.