In this paper, we present an information-theoretic approach to learning a Mahalanobis distance function. We formulate the problem as that of minimizing the differential relative entropy between two multivariate Gaussians under constraints on the distance function. We express this problem as a particular Bregman optimization problem---that of minimizing the LogDet divergence subject to linear constraints. Our resulting algorithm has several advantages over existing methods. First, our method can handle a wide variety of constraints and can optionally incorporate a prior on the distance function. Second, it is fast and scalable. Unlike most existing methods, no eigenvalue computations or semi-definite programming are required. We also present an online version and derive regret bounds for the resulting algorithm. Finally, we evaluate our method on a recent error reporting system for software called Clarify, in the context of metric learning for nearest neighbor classification, as well as on standard data sets

We develop geometric optimization on the manifold of Hermitian positive definite (HPD) matrices. In particular, we consider optimizing two types of cost functions: (i) geodesically convex (g-convex) and (ii) log-nonexpansive (LN). G-convex functions are nonconvex in the usual Euclidean sense but convex along the manifold and thus allow global optimization. LN functions may fail to be even g-convex but still remain globally optimizable due to their special structure. We develop theoretical tools to recognize and generate g-convex functions as well as cone theoretic fixed-point optimization algorithms. We illustrate our techniques by applying them to maximum-likelihood parameter estimation for elliptically contoured distributions (a rich class that substantially generalizes the multivariate normal distribution). We compare our fixed-point algorithms with sophisticated manifold optimization methods and obtain notable speedups.

Ultimately being motivated by facilitating space-variant blind deconvolution, we present a class of linear transformations, that are expressive enough for space-variant filters, but at the same time especially designed for efficient matrix-vector-multiplications. Successful results on astronomical imaging through atmospheric turbulences and on noisy magnetic resonance images of constantly moving objects demonstrate the practical significance of our approach.

Hermitian positive definite (hpd) matrices form a self-dual convex cone whose interior is a Riemannian manifold of nonpositive curvature. The manifold view comes with a natural distance function but the conic view does not. Thus, drawing motivation from convex optimization we introduce the S-divergence, a distance-like function on the cone of hpd matrices. We study basic properties of the S-divergence and explore its connections to the Riemannian distance. In particular, we show that (i) its square-root is a distance, and (ii) it exhibits numerous nonpositive-curvature-like properties.