How Active is Active Learning: Value Function Method Versus an Approximation Method
- PDF / 645,809 Bytes
- 19 Pages / 439.37 x 666.142 pts Page_size
- 117 Downloads / 174 Views
How Active is Active Learning: Value Function Method Versus an Approximation Method Hans M. Amman1
· Marco P. Tucci2
Accepted: 2 January 2020 © The Author(s) 2020
Abstract In a previous paper Amman et al. (Macroecon Dyn, 2018) compare the two dominant approaches for solving models with optimal experimentation (also called active learning), i.e. the value function and the approximation method. By using the same model and dataset as in Beck and Wieland (J Econ Dyn Control 26:1359–1377, 2002), they find that the approximation method produces solutions close to those generated by the value function approach and identify some elements of the model specifications which affect the difference between the two solutions. They conclude that differences are small when the effects of learning are limited. However the dataset used in the experiment describes a situation where the controller is dealing with a nonstationary process and there is no penalty on the control. The goal of this paper is to see if their conclusions hold in the more commonly studied case of a controller facing a stationary process and a positive penalty on the control. Keywords Optimal experimentation · Value function · Approximation method · Adaptive control · Active learning · Time-varying parameters · Numerical experiments JEL Classification C63 · E61 · E62
B
Marco P. Tucci [email protected] Hans M. Amman [email protected]
1
Faculty of Economics and Business, University of Amsterdam, Roetersstraat 11, 1018 WB Amsterdam, The Netherlands
2
Dipartimento di Economia Politica e Statistica, University of Siena, Piazza S. Francesco 7, 53100 Siena, Italy
123
H. M. Amman, M. P. Tucci
1 Introduction In recent years there has been a resurgent interest in economics on the subject of optimal or strategic experimentation also referred to as active learning, see e.g. Amman et al. (2018), Buera et al. (2011) and Savin and Blueschke (2016).1 There are two prevailing methods for solving this class of models. The first method is based on the value function approach and the second on an approximation method. The former uses dynamic programming for the full problem as used in studies by Prescott (1972), Taylor (1974), Easley and Kiefer (1988), Kiefer (1989), Kiefer and Nyarko (1989), Aghion et al. (1991) and more recently used in the work of Beck and Wieland (2002), Coenen et al. (2005), Levin et al. (2003) and Wieland (2000a, b). A nice set of applications on optimal experimentation, using the value function approach, can be found in Willems (2012). In principle, the value function approach should be the preferred method as it derives the optimal values for the policy variables through Bellman’s (1957) dynamic programming. Unfortunately, it suffers from the curse of dimensionality, as is shown in Bertsekas (1976). Hence, the value function approach is only applicable to small problems with one or two policy variables. This is caused by the fact that solution space needs to be discretized in such a fashion that it cannot be solved in feasible time. The approximation me
Data Loading...
