1 February 2003 in Neural Computation
Algorithms for data-driven learning of domain-specific overcomplete dictionaries are developed to obtain maximum likelihood and maximum a posteriori dictionary estimates based on the use of Bayesian models with concave/Schur-concave (CSC) negative log priors. Such priors are appropriate for obtaining sparse representations of environmental signals within an appropriately chosen (environmentally matched) dictionary. The elements of the dictionary can be interpreted as concepts, features, or words capable of succinct expression of events encountered in the environment (the source of the measured signals). This is a generalization of vector quantization in that one is interested in a description involving a few dictionary entries (the proverbial “25 words or less”), but not necessarily as succinct as one entry. To learn an environmentally adapted dictionary capable of concise expression of signals generated by the environment, we develop algorithms that iterate between a representative set of sparse representations found by variants of FOCUSS and an update of the dictionary using these sparse representations. Experiments were performed using synthetic data and natural images. For complete dictionaries, we demonstrate that our algorithms have improved performance over other independent component analysis (ICA) methods, measured in terms of signal-to-noise ratios of separated sources. In the overcomplete case, we show that the true underlying dictionary and sparse sources can be accurately recovered. In tests with natural images, learned overcomplete dictionaries are shown to have higher coding efficiency than complete dictionaries; that is, images encoded with an overcomplete dictionary have both higher compression (fewer bits per pixel) and higher accuracy (lower mean square error).
18 December 2020 in IEEE Transactions on Signal Processing
In this paper, we present a novel neural network architecture for MIMO symbol detection, the Recurrent Equivariant MIMO detector (RE-MIMO). It incorporates several important considerations in wireless communication systems, such as robustness to channel misspecification, the ability to handle a varying number of users with a single model, and invariance to the (sequential) order in which the users interact with the system. The decoder consists of three main blocks; one block to inform the decoder of the channel model, one permutation equivariant block based on the successful transformer architecture to model transmitter channel interactions, and one fully connected feedforward network to predict the demodulated symbols for each user. These blocks are chained together into an iterative decoder that is trained through end-to-end backpropagation. RE-MIMO is compared against a broad range of existing methods and the results confirm the ability of the network to achieve the state of the art demodulation accuracy. In particular, RE-MIMO efficiently handles a variable number of transmitters with only a single trained model, is capable of dealing with correlated channels, and is robust to channel misspecification. In terms of computational complexity, RE-MIMO scales favorably with the number of transmitters.
9 March 2018 in IEEE Journal of Selected Topics in Signal Processing
We address the problem of channel estimation in massive multiple-input multiple-output (Massive MIMO) systems where both hybrid analog-digital (AD) processing and low-resolution analog-to-digital converters (ADCs) are utilized. The hardware-efficient architecture is attractive from a power and cost point of view, but poses two significant channel estimation challenges. One is due to the smaller dimension of the measurement signal obtained from the limited number of RF chains, and the other is the coarser measurements from the low-resolution ADCs. We address this problem by utilizing two sources of information. Firstly, by exploiting the sparse nature of the channel in the angular domain, the channel estimate is enhanced and the required number of pilots is reduced. Secondly, by utilizing the transmitted data symbols as the "virtual pilots", the channel estimate is further improved without adding more pilot symbols. The constraints imposed by the architecture, the sparsity of the channel and the data aided channel estimation are treated in a unified manner by employing a Bayesian formulation. The quantized sparse channel estimation is formulated into a sparse Bayesian learning (SBL) framework, and solved using the variational Bayesian (VB) method. Simulation results show that the proposed algorithm can efficiently estimate the channel even with the architectural constraints, and that significant improvements are enabled by leveraging the transmitted data symbols.
10 March 2023 in IEEE Transactions on Signal Processing
In this work, we consider the parametric data model employed in applications such as line spectral estimation and direction-of-arrival estimation. We focus on the stochastic maximum likelihood estimation (MLE) framework and offer approaches to estimate the parameter of interest in a gridless manner, overcoming the model complexities of the past. This progress is enabled by the modern trend of reparameterization of the objective, and exploiting the sparse Bayesian learning (SBL) approach. The latter is shown to be a correlation-aware method, and for the underlying problem it is identified as a grid-based technique for recovering a structured covariance matrix of the measurements. For the case when the structured matrix is expressible as a sampled Toeplitz matrix, such as when measurements are sampled in time or space at regular intervals, additional constraints and reparameterization of the SBL objective leads to the proposed structured matrix recovery technique based on MLE. The proposed optimization problem is non-convex, and we propose a majorization-minimization based iterative procedure to estimate the structured matrix; each iteration solves a semidefinite program. We recover the parameter of interest in a gridless manner by appealing to the Carathéodory-Féjer result on decomposition of positive semidefinite (PSD) Toeplitz matrices. For the general case of irregularly spaced time or spatial samples, we propose an iterative SBL procedure that refines grid points to increase resolution near potential source locations, while maintaining a low per iteration complexity. We provide numerical results to evaluate and compare the performance of the proposed techniques with other gridless techniques, and the Cramér-Rao bound. The proposed correlation-aware approach is more robust to environmental/system effects such as low number of snapshots, correlated sources, small separation between source locations and improves sources identifiability.
9 January 2013 in IEEE Transactions on Information Theory
This paper studies the performance limits in the support recovery of sparse signals based on multiple measurement vectors (MMV). An information-theoretic analytical framework inspired by the connection to the single-input multiple-output multiple-access channel communication is established to reveal the performance limits in the support recovery of sparse signals with fixed number of nonzero entries. Sharp sufficient and necessary conditions for asymptotically successful support recovery are derived in terms of the number of measurements per vector, the number of nonzero rows, the measurement noise level, and the number of measurement vectors. Through the interpretations of the results, the benefit of having MMV for sparse signal recovery is illustrated, thus providing a theoretical foundation to the performance improvement enabled by MMV as observed in many existing simulation results. In particular, it is shown that the structure (rank) of the matrix formed by the nonzero entries plays an important role in the performance limits of support recovery.
9 November 2017 in IEEE Transactions on Signal Processing
Linear regression in the presence of outliers is an important problem and is challenging as the support of outliers is not known beforehand. Many robust estimators solve this problem via explicitly or implicitly assuming that outliers are sparse and result in large observation errors. We propose an Algorithm for Robust Outlier Support Identification (AROSI) utilizing a novel objective function with $\mathcal{l}_0$-‘norm’ regularization which models the sparsity of outliers. The optimization procedure naturally utilizes the large observation error assumption of outliers and directly operates on the $\mathcal{l}_0$-‘norm’ and is guaranteed to converge. When only sparse outliers are present (no dense inlier noise), we show that, under certain model and algorithm parameter settings, AROSI can recover the solution exactly. In the case where both dense inlier noise and sparse outliers are present, we prove that the estimation error is bounded. Extensive empirical comparisons with state-of-the-art methods demonstrate the advantage of the proposed method.