14 November 2018 in IEEE Signal Processing Magazine
In the era of big data, analysts usually explore various statistical models or machine-learning methods for observed data to facilitate scientific discoveries or gain predictive power. Whatever data and fitting procedures are employed, a crucial step is to select the most appropriate model or method from a set of candidates. Model selection is a key ingredient in data analysis for reliable and reproducible statistical inference or prediction, and thus it is central to scientific studies in such fields as ecology, economics, engineering, finance, political science, biology, and epidemiology. There has been a long history of model selection techniques that arise from researches in statistics, information theory, and signal processing. A considerable number of methods has been proposed, following different philosophies and exhibiting varying performances. The purpose of this article is to provide a comprehensive overview of them, in terms of their motivation, large sample performance, and applicability. We provide integrated and practically relevant discussions on theoretical properties of state-of-the-art model selection approaches. We also share our thoughts on some controversial views on the practice of model selection.
1 January 2000 in IEEE Transactions on Communications
21 July 2009 in IEEE Transactions on Wireless Communications
We study a cognitive network consisting of a single primary transmitter and multiple secondary, or cognitive, users. The primary transmitter, located at the center of the network, communicates with primary receivers within a disc called the primary exclusive region (PER). Inside the PER, no cognitive users may transmit, in order to guarantee an outage probability for the primary receivers within. Outside the PER, uniformly distributed cognitive users may transmit, provided they are at a certain protected radius from a primary receiver. We analyze the aggregated interference from the cognitive transmitters to a primary receiver within the PER. Based on this interference and the outage guarantee, we derive bounds on the radius of the PER, showing its interdependence on the receiver protected distance and other system parameters. We also extend the analysis to allowing the cognitive users to scale their power according to the distance from the primary transmitter. These studies provide a closed-form, theoretical analysis of such a network geometry with PER, which may be relevant in the upcoming spectrum sharing actions.
20 June 2017 in IEEE Transactions on Information Theory
To address order selection for an autoregressive model fitted to time series data, we propose a new information criterion. It has the benefits of the two well-known model selection techniques: the Akaike information criterion and the Bayesian information criterion. When the data are generated from a finite-order autoregression, the Bayesian information criterion is known to be consistent, and so is the new criterion. When the true order is infinity or suitably high with respect to the sample size, the Akaike information criterion is known to be efficient in the sense that its predictive performance is asymptotically equivalent to the best offered by the candidate models; in this case, the new criterion behaves in a similar manner. Different from the two classical criteria, the proposed criterion adaptively achieves either consistency or efficiency depending on the underlying true model. In practice, where the observed time series is given without any prior information about the model specification, the proposed order selection criterion is more flexible and reliable compared with classical approaches. Numerical results are presented, demonstrating the adaptivity of the proposed technique when applied to various data sets.
23 August 2017 in IEEE Transactions on Automatic Control
Distributed optimization increasingly plays a central role in economical and sustainable operation of cyber-physical systems. Nevertheless, the complete potential of the technology has not yet been fully exploited in practice due to communication limitations posed by the real-world infrastructures. This work investigates fundamental properties of distributed optimization based on gradient methods, where gradient information is communicated using a limited number of bits. In particular, a general class of quantized gradient methods are studied, where the gradient direction is approximated by a finite quantization set. Sufficient and necessary conditions are provided on such a quantization set to guarantee that the methods minimize any convex objective function with Lipschitz continuous gradient and a nonempty and bounded set of optimizers. A lower bound on the cardinality of the quantization set is provided, along with specific examples of minimal quantizations. Convergence rate results are established that connect the fineness of the quantization and the number of iterations needed to reach a predefined solution accuracy. Generalizations of the results to a relevant class of constrained problems using projections are considered. Finally, the results are illustrated by simulations of practical systems.