Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network
- PDF / 825,131 Bytes
- 7 Pages / 600.05 x 792 pts Page_size
- 37 Downloads / 211 Views
Research Article Multiobjective Reinforcement Learning for Traffic Signal Control Using Vehicular Ad Hoc Network Duan Houli, Li Zhiheng, and Zhang Yi Department of Automation, Tsinghua University, Beijing 100084, China Correspondence should be addressed to Duan Houli, [email protected] Received 1 December 2009; Accepted 5 September 2010 Academic Editor: Hossein Pishro-Nik Copyright © 2010 Duan Houli et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited. We propose a new multiobjective control algorithm based on reinforcement learning for urban traffic signal control, named multiRL. A multiagent structure is used to describe the traffic system. A vehicular ad hoc network is used for the data exchange among agents. A reinforcement learning algorithm is applied to predict the overall value of the optimization objective given vehicles’ states. The policy which minimizes the cumulative value of the optimization objective is regarded as the optimal one. In order to make the method adaptive to various traffic conditions, we also introduce a multiobjective control scheme in which the optimization objective is selected adaptively to real-time traffic states. The optimization objectives include the vehicle stops, the average waiting time, and the maximum queue length of the next intersection. In addition, we also accommodate a priority control to the buses and the emergency vehicles through our model. The simulation results indicated that our algorithm could perform more efficiently than traditional traffic light control methods.
1. Introduction Increasing traffic congestion over the road networks makes the development of more intelligent and efficient traffic control systems an urgent and important requirement. However, traffic systems are typically complex large-scale systems consisting of a great number of interacting participants. It is very difficult to use traditional control algorithms to get satisfied control effect. Thus, various intelligent algorithms have been used in attempts to build an efficient traffic control system, such as fuzzy control technologies [1, 2], artificial neural networks [3, 4], and genetic algorithms [5, 6], which greatly improve the efficiency of urban traffic signal control systems. Reinforcement learning is a category of machine learning algorithms including Q learning, temporal difference, and SARSA algorithm [7–9]. Reinforcement learning is to learn the optimal policy by a trial-and-error process including perceiving states from the environment, choosing an action according to current states and receiving rewards from the environment. The policy which maximizes the expected
long-term cumulative reward is considered as the optimal one. Reinforcement learning is a self-learning algorithm which does not need an explicit model of the environment. Thus, it can be applied in traffic signal control effectively to respond to the constant changes of traffic flow and
Data Loading...
