Distributed MPC Using Reinforcement Learning Based Negotiation: Application to Large Scale Systems
This chapter describes a methodology to deal with the interaction (negotiation) between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order
- PDF / 623,873 Bytes
- 17 Pages / 439.37 x 666.142 pts Page_size
- 113 Downloads / 193 Views
Distributed MPC Using Reinforcement Learning Based Negotiation: Application to Large Scale Systems B. Morcego, V. Javalera, V. Puig and R. Vito
Abstract This chapter describes a methodology to deal with the interaction (negotiation) between MPC controllers in a distributed MPC architecture. This approach combines ideas from Distributed Artificial Intelligence (DAI) and Reinforcement Learning (RL) in order to provide a controller interaction based on negotiation, cooperation and learning techniques. The aim of this methodology is to provide a general structure to perform optimal control in networked distributed environments, where multiple dependencies between subsystems are found. Those dependencies or connections often correspond to control variables. In that case, the distributed control has to be consistent in each subsystem. One of the main new concepts of this architecture is the negotiator agent. Negotiator agents interact with MPC agents to reach an agreement on the optimal value of the shared control variables. The optimal value of those shared control variables has to accomplish a common goal, probably incompatible with the specific goals of each partition that share the variable. Two cases of study are discussed, a small water distribution network and the Barcelona water network. The results suggest that this approach is a promising strategy when centralized control is not a reasonable choice.
32.1 Introduction When dealing with LSS and distributed MPC approaches to control them, one of the main concerns is to handle the interactions between subsystems. The goal of the research described in this chapter is to propose and study negotiation methods to solve that problem. Those negotiation methods are based on learning techniques, namely Q-learning, which emerges from the RL theory. That way, it is possible to exploit the attractive features of MPC, such as meaningful objective functions and B. Morcego (B) · V. Javalera · V. Puig · R. Vito Advanced Control Systems Group, Terrassa, Spain e-mail: [email protected] J. M. Maestre and R. R. Negenborn (eds.), Distributed Model Predictive Control 517 Made Easy, Intelligent Systems, Control and Automation: Science and Engineering 69, DOI: 10.1007/978-94-007-7006-5_32, © Springer Science+Business Media Dordrecht 2014
518
B. Morcego et al.
constraints, in a distributed implementation combining learning techniques to perform the negotiation of the variables in a cooperative Multi Agent (MA) environment and over a Multi Agent platform. Negotiation between distributed controllers in LSS is an open issue. Conventional negotiation techniques are not suitable for many reasons: calculation time, problems handling multiple restrictions and multiple objectives and the impossibility to ensure convergence are the most common reasons. Although there are successful results, there is a need of a methodology that can be used for all kinds of continuous LSS. One of the most accepted techniques is the augmented Lagrangian method. The seminal Tamura coordination method was
Data Loading...
