Mirror Descent Algorithms for Minimizing Interacting Free Energy
- PDF / 431,459 Bytes
- 14 Pages / 439.37 x 666.142 pts Page_size
- 45 Downloads / 175 Views
Mirror Descent Algorithms for Minimizing Interacting Free Energy Lexing Ying1 Received: 13 April 2020 / Revised: 18 July 2020 / Accepted: 2 September 2020 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract This note considers the problem of minimizing interacting free energy. Motivated by the mirror descent algorithm, for a given interacting free energy, we propose a descent dynamics with a novel metric that takes into consideration the reference measure and the interacting term. This metric naturally suggests a monotone reparameterization of the probability measure. By discretizing the reparameterized descent dynamics with the explicit Euler method, we arrive at a new mirror-descent-type algorithm for minimizing interacting free energy. Numerical results are included to demonstrate the efficiency of the proposed algorithms. Keywords Mirror descent algorithms · Interacting free energy · Kullback–Leibler divergence · Reverse Kullback—Leibler divergence · Hellinger divergence
1 Introduction This paper considers the problem of minimizing free energies of the following form 1 p(x)W (x, y) p(y)dxdy F( p) = D( p||μ) + p(x)V (x)dx + 2
(1)
for a probability density p over domain . D( p||μ) is a divergence function between p and a reference density μ and typically examples are Kullback–Leibler divergence, reverse Kullback–Leibler divergence, and Hellinger divergence. In the interacting term pW pdxdy, W is symmetric and can either be positive-definite or not. Non-positivedefinite interacting terms appear in Keller–Segel models in mathematical biology and granular flows in kinetic theory. Recently, positive-definite interacting terms appear in the mean field modeling of neural network training [9,15,19,21].
The work of L.Y. is partially supported by the U.S. Department of Energy, Office of Science, Office of Advanced Scientific Computing Research, Scientific Discovery through Advanced Computing (SciDAC) program and also by the National Science Foundation under award DMS-1818449. The author thanks Wuchen Li and Wotao Yin for comments and suggestions. Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.
B 1
Lexing Ying [email protected] Department of Mathematics and ICME, Stanford University, Stanford, CA 94305, USA 0123456789().: V,-vol
123
51
Page 2 of 14
Journal of Scientific Computing
(2020) 84:51
The goal of this paper is to develop fast first-order algorithms for identifying minimums of (1). When F is convex (for example, when W is positive-definite), there exists a unique global minimizer and the goal is to compute this global minimizer efficiently. When F is non-convex, there are typically many local minimums and the more moderate goal is to find one such local minimum. There are several difficulties for computing local minima for (1). First, this is an optimization problem over probability simplex, hence one needs to deal with the constraints p(x) ≥ 0 and p(x)dx = 1. Second, when the reference measure μ(x)
Data Loading...
