Robust Deep Reinforcement Learning for Traffic Signal Control
- PDF / 1,004,020 Bytes
- 12 Pages / 595.276 x 790.866 pts Page_size
- 57 Downloads / 234 Views
ORIGINAL PAPER
Robust Deep Reinforcement Learning for Traffic Signal Control Kai Liang Tan1 · Anuj Sharma2 · Soumik Sarkar1 Received: 13 August 2020 / Revised: 29 October 2020 / Accepted: 23 November 2020 © The Author(s), under exclusive licence to Springer Nature Singapore Pte Ltd. part of Springer Nature 2020
Abstract A traffic signal is a fundamental part of the traffic control system to reduce congestion and enhance safety. Since the inception of motorized vehicles, traffic signal controllers are put in place to coordinate and maintain traffic flow. With the number of vehicles on the road increasing exponentially, it is imperative to innovate new traffic control frameworks to cope with the high-density traffic demand. In this regard, recent advances in machine/deep learning have enabled significant progress towards reducing congestion using reinforcement learning for traffic signal control. However, most of these works are still not ready for deployment due to assumptions of perfect knowledge of the traffic environment. In reality, congestion detection or prediction systems are at best able to approximate the traffic state with significant noise. In this work, we propose a robust training framework for reinforcement learning agents that can handle such noisy approximation of the traffic states. Specifically, we show that by carefully adding synthetic perturbations to the state space, such as the queue length during training, the reinforcement learning agents can be robustified. Conceptually, our approach is similar to adversarial training schemes and can lead to successful deployment of reinforcement learning agent-based traffic signal controllers. Keywords Deep learning · Deep reinforcement learning · Robust · Adaptive signal control technologies
Introduction Transportation is an integral part of our lives to commute from one place to another. As vehicles became more affordable to the public, the number of congestion also increased linearly with the number of cars sold. A study by Texas A&M Transportation Institute on the effects of congestion since 1982 showed overwhelmingly negative effects on our society (Lasley 2019). For 35 years, the total traffic delay increased by 8.8 billion hours, fuel waste increased by 3.35 billion gallons, which equates to a total congestion cost of 179 billion dollars. Hence, it is imperative to introduce
* Soumik Sarkar [email protected] Kai Liang Tan [email protected] Anuj Sharma [email protected] 1
Department of Mechanical Engineering, Iowa State University, Ames, Iowa 50011, USA
Department of Civil Engineering, Iowa State University, Ames, Iowa 50011, USA
2
newer Adaptive Signal Control Technologies (ASCT) methods to reduce the number of congestion drastically. Traditionally, the traffic control problem is formulated as an optimization problem, where certain unrealistic assumptions are necessary to make the problem tractable (Roess et al. 2004). With these assumptions, a traffic engineer crafts a timing plan based on historical traffic volumes for a specif
Data Loading...
