A new method based on the proximal bundle idea and gradient sampling technique for minimizing nonsmooth convex functions
- PDF / 2,775,043 Bytes
- 31 Pages / 439.37 x 666.142 pts Page_size
- 6 Downloads / 207 Views
A new method based on the proximal bundle idea and gradient sampling technique for minimizing nonsmooth convex functions M. Maleknia1 · M. Shamsi1 Received: 15 November 2019 © Springer Science+Business Media, LLC, part of Springer Nature 2020
Abstract In this paper, we combine the positive aspects of the gradient sampling (GS) and bundle methods, as the most efficient methods in nonsmooth optimization, to develop a robust method for solving unconstrained nonsmooth convex optimization problems. The main aim of the proposed method is to take advantage of both GS and bundle methods, meanwhile avoiding their drawbacks. At each iteration of this method, to find an efficient descent direction, the GS technique is utilized for constructing a local polyhedral model for the objective function. If necessary, via an iterative improvement process, this initial polyhedral model is improved by some techniques inspired by the bundle and GS methods. The convergence of the method is studied, which reveals that the global convergence property of our method is independent of the number of gradient evaluations required to establish and improve the initial polyhedral models. Thus, the presented method needs much fewer gradient evaluations in comparison to the original GS method. Furthermore, by means of numerical simulations, we show that the presented method provides promising results in comparison with GS methods, especially for large scale problems. Moreover, in contrast with some bundle methods, our method is not very sensitive to the accuracy of supplied gradients. Keywords Nonsmooth convex optimization · Unconstrained minimization · Gradient sampling · Bundle method Mathematics Subject Classification 65K05 · 90C26 · 49M05 · 49M37
* M. Shamsi [email protected] M. Maleknia [email protected] 1
Amirkabir University of Technology, Tehran, Iran
13
Vol.:(0123456789)
M. Maleknia, M. Shamsi
1 Introduction In this paper, we consider the following minimization problem
min f (𝐱)
s.t.
𝐱 ∈ ℝn ,
(1)
where f ∶ ℝn → ℝ is a convex function, but not necessarily differentiable. Such problems appear in many applied fields, such as data analysis, image processing, optimal control, biology, chemistry, and computational physics [2, 20, 24, 25, 32]. In this regard, it is worthwhile to develop efficient methods for solving these types of problems. There are various methods for locating a minimizer of problem (1). The subgradient method, initially developed by Shor[30], is one of the simplest methods for solving a nonsmooth convex optimization problem. Because of its simple structure, it is still a widely used method, although it suffers from some serious drawbacks, such as lack of descent, slow convergence, and lack of practical termination criterion. As another class of methods for solving nonsmooth optimization problems, we can refer to bundle methods as one of the most efficient methods in nonsmooth optimization, which can handle both convex and nonconvex objectives. One drawback of the bundle type methods was that, at each i
Data Loading...
