This paper addresses the problem of downlink channel estimation in frequency-division duplexing massive multiple-input multiple-output systems. The existing methods usually exploit hidden sparsity under a discrete Fourier transform (DFT) basis to estimate the downlink channel. However, there are at least two shortcomings of these DFT-based methods: first, they are applicable to uniform linear arrays (ULAs) only, since the DFT basis requires a special structure of ULAs; and second, they always suffer from a performance loss due to the leakage of energy over some DFT bins. To deal with the above-mentioned shortcomings, we introduce an off-grid model for downlink channel sparse representation with arbitrary two-dimensional-array antenna geometry, and propose an efficient sparse Bayesian learning approach for the sparse channel recovery and off-grid refinement. The main idea of the proposed off-grid method is to consider the sampled grid points as adjustable parameters. Utilizing an in-exact block majorization-minimization algorithm, the grid points are refined iteratively to minimize the off-grid gap. Finally, we further extend the solution to uplink-aided channel estimation by exploiting the angular reciprocity between downlink and uplink channels, which brings enhanced recovery performance.

Conventional direction-of-arrival (DOA) estimation methods are sensitive to outlier measurements. Therefore, their performance may degrade substantially in the presence of impulsive noise. In this paper, we address the problem of DOA estimation in additive outliers from the perspective of sparse Bayesian learning (SBL). A Bayes-optimal algorithm is devised for robust DOA estimation, which can achieve excellent performance in terms of resolution and accuracy. To reduce the computational complexity of the SBL scheme, a fast alternating algorithm is also developed. New grid-refining procedures are further introduced into these two proposed algorithms to efficiently fix the off-grid gap. As our solutions do not require the prior knowledge of the number of sources and can resolve highly correlated or coherent sources, it is expected that they have higher applicability. Simulation results verify the outlier-robust performance of the SBL approach.

Sparse Bayesian learning (SBL) has become a popular approach for direction-of-arrival (DOA) estimation, but its computational complexity for Bayesian inference is quite high because calculating inverse of a large complex matrix per iteration is required. It is known that the computational load can be reduced by transforming the complex-valued problem into a real-valued one. However, the commonly used real-valued transformation works for uniform linear arrays (ULAs) only. In this paper, we propose a new real-valued transformation for DOA estimation with arbitrary linear arrays by exploiting the virtual steering of linear arrays. Then, we introduce an alternating optimization algorithm based on the variational Bayesian inference (VBI) methodology to iteratively obtain a stationary solution to the real-valued sparse representation problem. Because of utilizing the additional real-valued structure, the VBI scheme can achieve a better performance in terms of both estimation accuracy and computational complexity. Moreover, we embed the generalized approximate message passing (GAMP) into the VBI-based method for further complexity reduction. Although there may be a performance loss for the GAMP variant, simulation results reveal its substantial performance improvement over existing methods.

We address the downlink channel estimation problem for massive multiple-input multiple-output (MIMO) systems in this paper, where the inherit burst-sparsity structure is exploited to improve the channel estimation performance. In the literature, the commonly used burst-sparsity model assumes a uniform burst-sparse structure in which all bursts have similar sizes. However, such assumption is oversimplified to hold in practice. Outliers deviated from such uniform burst structures can significantly degrade the accuracy of the existing burst-sparsity models, which may result in a reduced recovery performance. To capture a more general burst-sparsity structure in practice, we propose a novel non-uniform burst-sparsity model and introduce an improved pattern-coupled prior to account for more realistic non-uniform burst structures. A generic sparse Bayesian learning based framework to exploit the non-uniform burst-sparsity and to enhance massive MIMO channel estimation performance is then developed. We further prove that our solution converges to a stationary point of the associated optimization problem, and our framework includes the state-of-the-art pattern-coupled method as a special case. Simulation results verify the robust performance of the devised method.