Specialization in Hierarchical Learning Systems
- PDF / 4,435,863 Bytes
- 34 Pages / 439.37 x 666.142 pts Page_size
- 97 Downloads / 208 Views
Specialization in Hierarchical Learning Systems A Unified Information-theoretic Approach for Supervised, Unsupervised and Reinforcement Learning Heinke Hihn1
· Daniel A. Braun1
Accepted: 5 September 2020 © The Author(s) 2020
Abstract Joining multiple decision-makers together is a powerful way to obtain more sophisticated decision-making systems, but requires to address the questions of division of labor and specialization. We investigate in how far information constraints in hierarchies of experts not only provide a principled method for regularization but also to enforce specialization. In particular, we devise an information-theoretically motivated on-line learning rule that allows partitioning of the problem space into multiple sub-problems that can be solved by the individual experts. We demonstrate two different ways to apply our method: (i) partitioning problems based on individual data samples and (ii) based on sets of data samples representing tasks. Approach (i) equips the system with the ability to solve complex decision-making problems by finding an optimal combination of local expert decision-makers. Approach (ii) leads to decision-makers specialized in solving families of tasks, which equips the system with the ability to solve meta-learning problems. We show the broad applicability of our approach on a range of problems including classification, regression, density estimation, and reinforcement learning problems, both in the standard machine learning setup and in a meta-learning setting. Keywords Meta-learning · Information theory · Bounded rationality
1 Introduction Intelligent agents are often conceptualized as decision-makers that learn probabilistic models of their environment and optimize utilities or disutilities like cost or loss functions [91]. In the general case we can think of a utility function as a black-box oracle that provides a numerical score that rates any proposed solution to a supervised, unsupervised or reinforcement learning problem. In context of decision-making, naïvely enumerating all possibilities
B
Heinke Hihn [email protected] Daniel A. Braun [email protected]
1
Institute for Neural Information Processing, Ulm University, Ulm, Germany
123
H. Hihn, D. A. Braun
and searching for an optimal solution is usually prohibitively expensive. Instead, intelligent agents must invest their limited resources in such a way that they achieve an optimal trade-off between expected utility and resource costs in order to enable efficient learning and acting. This trade-off is the central issue in the fields of bounded or computational rationality with repercussions across other disciplines including economics, psychology, neuroscience and artificial intelligence [3,16,23,24,26,35,55,66,78]. The information-theoretic approach to bounded rationality is a particular instance of bounded rationality where the resource limitations are modeled by information constraints [18,28,54,59,63,64,73,85,92] closely related to Jaynes’ maximum entropy or minimum relative entropy principle
Data Loading...
