Online reliability optimization for URLLC in HetNets: a DQN approach
- PDF / 2,729,544 Bytes
- 20 Pages / 595.276 x 790.866 pts Page_size
- 3 Downloads / 171 Views
(0123456789().,-volV)(0123456789(). ,- volV)
ORIGINAL ARTICLE
Online reliability optimization for URLLC in HetNets: a DQN approach Leyou Yang1,2 • Jie Jia1,2,3
•
Jian Chen1,4 • Xingwei Wang1,2
Received: 5 February 2020 / Accepted: 27 October 2020 Ó Springer-Verlag London Ltd., part of Springer Nature 2020
Abstract Heterogeneous cellular networks (HetNets) have been proven as a promising approach to deal with ever-growing data traffic. Supporting ultra-reliable and low-latency communication (URLLC) is also considered as a new feature of the upcoming wireless networks. Due to the overlapping structure and the mutual interference between cells in HetNets, existing resource allocation approaches cannot be directly applied for real-time applications, especially for URLLC services. As a novel unsupervised algorithm, Deep Q Network (DQN) has already been applied to many online complex optimization models successfully. However, it may perform badly for resource allocation optimization in HetNets, due to the tiny state change and the large-scale action space characteristics. In order to cope with them, we first propose an autoencoder to disturb the similarity of adjacent states to enhance the features and then divide the whole decision process into two phases. DQN is applied to solve each phase, respectively, and we iterate the whole process to find the joint optimized solution. We implement our algorithm in 6 scenarios with different numbers of user equipment (UE), redundant links, and sub-carriers. Simulations results demonstrate that our algorithm has good convergence for the optimization objective. Moreover, by further optimizing the power allocation, a 1–2 nines of reliability improvement is obtained for bad conditions. Finally, the experiment result shows that our algorithm reaches the reliability of 8-nines in common scenarios. As an online method, the algorithm proposed in this paper takes only 0.32 s on average. Keywords Deep Q network HetNets Online optimization URLLC
1 Introduction Over the last decades, the world has witnessed the evolution of mobile wireless communication techniques. Due to the penetration of smartphones, tablets, and machine-tomachine communication devices, mobile data traffic grows exponentially. It is reported that global mobile traffic will & Jie Jia [email protected] 1
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China
2
Engineering Research Center of Security Technology of Complex Network System, Ministry of Education, Shenyang 110819, China
3
Key Laboratory of Intelligent Internet Theory and Application, Liaoning Province, Northeastern University, Shenyang 110819, China
4
Liaoning Research Center of Safety Engineering Technology in Industrial Control, Neusoft Group Research, Shenyang 110179, China
reach 77 exabytes per month by 2022, according to Cisco’s forecasts [10]. Like previous communications technology, 5G breeds a variety of new applications and services such as autonomous driving, haptic communications, and virtu
Data Loading...
