Event-based optimization with random packet dropping
- PDF / 671,214 Bytes
- 13 Pages / 595 x 842 pts (A4) Page_size
- 32 Downloads / 168 Views
. RESEARCH PAPER .
November 2020, Vol. 63 212202:1–212202:13 https://doi.org/10.1007/s11432-019-2702-x
Event-based optimization with random packet dropping Qing-Shan JIA1* , Jing-Xian TANG1 & Zhenning LANG2 1
Center for Intelligent and Networked Systems, Department of Automation, Tsinghua University, Beijing 100084, China; 2 NetEase Inc., Beijing 100089, China
Received 6 May 2019/Revised 1 August 2019/Accepted 27 September 2019/Published online 9 October 2020
Abstract Event-based optimization (EBO) provides a general framework for policy optimization in many discrete event dynamic systems where decision making is triggered by events that represent state transitions with common features. Because the number of events can be defined by the user and usually increases linearly with respect to the system scale, EBO has good potential to address large-scale problems where the state space grows exponentially. However, in many practical systems, sensors are geographically distributed and connected to a central controller through imperfect communication channels. Therefore, events observed by the sensors may not reach the controller. Optimization methods of event-based policies in these cases have not yet been identified. In this paper, we consider this important problem and make three major contributions. First, we formulate a mathematical EBO model in which the communication between sensors and controllers is subject to random packet dropping. Second, we show that this EBO model can be converted to another EBO model with perfect communication. Then, the performance difference equation and the performance derivative equation for event-based policies are straightforward to develop. One gradient-based policy iteration algorithm is developed for problems where the state transition probabilities are explicitly known, while another for problems where they are unknown. Third, the performance of the algorithms and the impact of the packet dropping probability on policy performance are numerically demonstrated on a single-zone occupant level estimation problem in buildings. Keywords
event-based optimization, packet dropping, discrete event dynamic systems
Citation Jia Q-S, Tang J-X, Lang Z N. Event-based optimization with random packet dropping. Sci China Inf Sci, 2020, 63(11): 212202, https://doi.org/10.1007/s11432-019-2702-x
1
Introduction
The discrete event dynamic system (DEDS) [1] has provided a general model for many man-made systems where the dynamics follow man-made rules rather than just physical laws. In this system, the state transition is triggered by events and not by time. It is well known that performance optimization in DEDS can be formulated as a generalized semi-Markov decision process (GSMDP), which can be converted to a Markov decision process (MDP) through appropriately defining and extending the state space. However, in general, solving a large-scale MDP is difficult because of two issues known as curses. The first is the curse of dimensionality. It refers to the fact that the state space may grow ex
Data Loading...
