Maximum likelihood estimation in finite mixture distributions is typically approached as an incomplete data problem to allow application of the expectation-maximization (EM) algorithm. In its general formulation, the EM algorithm involves the notion of a complete data space, in which the observed measurements and incomplete data are embedded. An advantage is that many difficult estimation problems are facilitated when viewed in this way. One drawback is that the simultaneous update used by standard EM requires overly informative complete data spaces, which leads to slow convergence in some situations. In the incomplete data context, it has been shown that the use of less informative complete data spaces, or equivalently smaller missing data spaces, can lead to faster convergence without sacrifying simplicity. However, in the mixture case, little progress has been made in speeding up EM. In this article we propose a component-wise EM for mixtures. It uses, at each iteration, the smallest admissible missing...

Accelerated algorithms for maximum-likelihood image reconstruction are essential for emerging applications such as three-dimensional (3-D) tomography, dynamic tomographic imaging, and other high-dimensional inverse problems. In this paper, we introduce and analyze a class of fast and stable sequential optimization methods for computing maximum-likelihood estimates and study its convergence properties. These methods are based on a proximal point algorithm implemented with the Kullback-Liebler (KL) divergence between posterior densities of the complete data as a proximal penalty function. When the proximal relaxation parameter is set to unity, one obtains the classical expectation-maximization (EM) algorithm. For a decreasing sequence of relaxation parameters, relaxed versions of EM are obtained which can have much faster asymptotic convergence without sacrifice of monotonicity. We present an implementation of the algorithm using More's (1983) trust region update strategy. For illustration, the method is applied to a nonquadratic inverse problem with Poisson distributed data.

In this paper, we analyze the celebrated EM algorithm from the point of view of proximal point algorithms. More precisely, we study a new type of generalization of the EM procedure introduced in [Chretien and Hero (1998)] and called Kullback-proximal algorithms. The proximal framework allows us to prove new results concerning the cluster points. An essential contribution is a detailed analysis of the case where some cluster points lie on the boundary of the parameter space.

We address the issue of estimating the regression vector β in the generic s-sparse linear model y = Xβ + z, with β ∈ ℝ ^p , y ∈ ℝ ⁿ , z ~ )V (0, σ ² I), and p > n when the variance σ2 is unknown. We study two least absolute shrinkage and selection operator (LASSO)-type methods that jointly estimate β and the variance. These estimators are minimizers of the l1 penalized least-squares functional, where the relaxation parameter is tuned according to two different strategies. In the first strategy, the relaxation parameter is of the order σ̂√log p, where σ̂ ² is the empirical variance. In the second strategy, the relaxation parameter is chosen so as to enforce a tradeoff between the fidelity and the penalty terms at optimality. For both estimators, our assumptions are similar to the ones proposed by Candès and Plan in Ann. Stat. (2009), for the case where σ ² is known. We prove that our estimators ensure exact recovery of the support and sign pattern of β with high probability. We present simulation results showing that the first estimator enjoys nearly the same performances in practice as the standard LASSO (known variance case) for a wide range of the signal-to-noise ratio. Our second estimator is shown to outperform both in terms of false detection, when the signal-to-noise ratio is low.