24 August 2021 in Pure and Applied Geophysics
19 May 2022 in SIAM Journal on Optimization
We develop a trust-region method for minimizing the sum of a smooth term (f) and a nonsmooth term (h), both of which can be nonconvex. Each iteration of our method minimizes a possibly nonconvex mo...
26 December 2022 in Mathematical Programming
12 August 2024 in SIAM Journal on Scientific Computing
We develop a Levenberg–Marquardt method for minimizing the sum of a smooth nonlinear least-squares term [Formula: see text] and a nonsmooth term [Formula: see text]. Both [Formula: see text] and [Formula: see text] may be nonconvex. Steps are computed by minimizing the sum of a regularized linear least-squares model and a model of [Formula: see text] using a first-order method such as the proximal gradient method. We establish global convergence to a first-order stationary point under the assumptions that [Formula: see text] and its Jacobian are Lipschitz continuous and [Formula: see text] is proper and lower semicontinuous. In the worst case, our method performs [Formula: see text] iterations to bring a measure of stationarity below [Formula: see text]. We also derive a trust-region variant that enjoys similar asymptotic worst-case iteration complexity as a special case of the trust-region algorithm of Aravkin, Baraldi, and Orban [ SIAM J. Optim., 32 (2022), pp. 900–929]. We report numerical results on three examples: a group-lasso basis-pursuit denoise example, a nonlinear support vector machine, and parameter estimation in a neuroscience application. To implement those examples, we describe in detail how to evaluate proximal operators for separable [Formula: see text] and for the group lasso with trust-region constraint. In all cases, the Levenberg–Marquardt methods perform fewer outer iterations than either a proximal gradient method with adaptive step length or a quasi-Newton trust-region method, neither of which exploit the least-squares structure of the problem. Our results also highlight the need for more sophisticated subproblem solvers than simple first-order methods.
7 October 2019 in IEEE Transactions on Signal Processing
Level-set optimization formulations with data-driven constraints minimize a regularization functional subject to matching observations to a given error level. These formulations are widely used, particularly for matrix completion and sparsity promotion in data interpolation and denoising. The misfit level is typically measured in the $\ell_2$ norm, or other smooth metrics.
21 February 2024 in Optimization Letters