Tensors or multiway arrays are functions of three or more indices (i, j, k, . . . )-similar to matrices (two-way arrays), which are functions of two indices (r, c) for (row, column). Tensors have a rich history, stretching over almost a century, and touching upon numerous disciplines; but they have only recently become ubiquitous in signal and data analytics at the confluence of signal processing, statistics, data mining, and machine learning. This overview article aims to provide a good starting point for researchers and practitioners interested in learning about and working with tensors. As such, it focuses on fundamentals and motivation (using various application examples), aiming to strike an appropriate balance of breadth and depth that will enable someone having taken first graduate courses in matrix algebra and probability to get started doing research and/or developing tensor algorithms and software. Some background in applied optimization is useful but not strictly required. The material covered includes tensor rank and rank decomposition; basic tensor factorization models and their relationships and properties (including fairly good coverage of identifiability); broad coverage of algorithms ranging from alternating optimization to stochastic gradient; statistical performance analysis; and applications ranging from source separation to collaborative filtering, mixture and topic modeling, classification, and multilinear subspace learning.
23 August 2018 in IEEE Transactions on Signal Processing
Numerical optimization has played a central role in addressing key signal processing (SP) problems. Highly effective methods have been developed for a large variety of SP applications such as communications, radar, filter design, and speech and image analytics, just to name a few. However, optimization algorithms often entail considerable complexity, which creates a serious gap between theoretical design/analysis and real-time processing. In this paper, we aim at providing a new learning-based perspective to address this challenging issue. The key idea is to treat the input and output of an SP algorithm as an unknown nonlinear mapping and use a deep neural network (DNN) to approximate it. If the nonlinear mapping can be learned accurately by a DNN of moderate size, then SP tasks can be performed effectively-since passing the input through a DNN only requires a small number of simple operations. In our paper, we first identify a class of optimization algorithms that can be accurately approximated by a fully connected DNN. Second, to demonstrate the effectiveness of the proposed approach, we apply it to approximate a popular interference management algorithm, namely, the WMMSE algorithm. Extensive experiments using both synthetically generated wireless channel data and real DSL channel data have been conducted. It is shown that, in practice, only a small network is sufficient to obtain high approximation accuracy, and DNNs can achieve orders of magnitude speedup in computational time compared to the state-of-the-art interference management algorithm.
In this article, an overview of advanced convex optimization approaches to multisensor beamforming is presented, and connections are drawn between different types of optimization-based beamformers that apply to a broad class of receive, transmit, and network beamformer design problems. It is demonstrated that convex optimization provides an indispensable set of tools for beamforming, enabling rigorous formulation and effective solution of both long-standing and emerging design problems.
26 February 2019 in IEEE Signal Processing Magazine
Detection and estimation problems in multiple-input multiple-output (MIMO) radar have recently drawn considerable interest in the signal processing community. Radar has long been a staple of signal processing, and MIMO radar presents challenges and opportunities in adapting classical radar imaging tools and developing new ones. Our aim in this article is to showcase the potential of tensor algebra and multidimensional harmonic retrieval (HR) in signal processing for MIMO radar. Tensor algebra and multidimensional HR are relatively mature topics, albeit still on the fringes of signal processing research. We show they are in fact central for target localization in a variety of pertinent MIMO radar scenarios. Tensor algebra naturally comes into play when the coherent processing interval comprises multiple pulses, or multiple transmit and receive subarrays are used (multistatic configuration). Multidimensional harmonic structure emerges for far-field uniform linear transmit/receive array configurations, also taking into account Doppler shift; and hybrid models arise in-between. This viewpoint opens the door for the application and further development of powerful algorithms and identifiability results for MIMO radar. Compared to the classical radar-imaging-based methods such as Capon or MUSIC, these algebraic techniques yield improved performance, especially for closely spaced targets, at modest complexity.
11 October 2013 in IEEE Transactions on Signal Processing
Non-negative matrix factorization (NMF) has found numerous applications, due to its ability to provide interpretable decompositions. Perhaps surprisingly, existing results regarding its uniqueness properties are rather limited, and there is much room for improvement in terms of algorithms as well. Uniqueness aspects of NMF are revisited here from a geometrical point of view. Both symmetric and asymmetric NMF are considered, the former being tantamount to element-wise non-negative square-root factorization of positive semidefinite matrices. New uniqueness results are derived, e.g., it is shown that a sufficient condition for uniqueness is that the conic hull of the latent factors is a superset of a particular second-order cone. Checking this condition is shown to be NP-complete; yet this and other results offer insights on the role of latent sparsity in this context. On the computational side, a new algorithm for symmetric NMF is proposed, which is very different from existing ones. It alternates between Procrustes rotation and projection onto the non-negative orthant to find a non-negative matrix close to the span of the dominant subspace. Simulation results show promising performance with respect to the state-of-art. Finally, the new algorithm is applied to a clustering problem for co-authorship data, yielding meaningful and interpretable results.