8 June 2018 in Journal of the American Statistical Association
The Hájek–Feldman dichotomy establishes that two Gaussian measures are either mutually absolutely continuous with respect to each other (and hence there is a Radon–Nikodym density for each measure with respect to the other one) or mutually singular. Unlike the case of finite-dimensional Gaussian measures, there are nontrivial examples of both situations when dealing with Gaussian stochastic processes. This article provides: (a) Explicit expressions for the optimal (Bayes) rule and the minimal classification error probability in several relevant problems of supervised binary classification of mutually absolutely continuous Gaussian processes. The approach relies on some classical results in the theory of reproducing kernel Hilbert spaces (RKHS). (b) An interpretation, in terms of mutual singularity, for the so-called “near perfect classification” phenomenon. We show that the asymptotically optimal rule proposed by these authors can be identified with the sequence of optimal rules for an approximating sequence of classification problems in the absolutely continuous case. (c) As an application, we discuss a natural variable selection method, which essentially consists of taking the original functional data X(t), t ∈ [0, 1] to a d-dimensional marginal (X(t1), …, X(td)), which is chosen to minimize the classification error of the corresponding Fisher’s linear rule. We give precise conditions under which this discrimination method achieves the minimal classification error of the original functional problem. Supplementary materials for this article are available online.
28 April 2015 in Journal of Statistical Computation and Simulation
The use of variable selection methods is particularly appealing in statistical problems with functional data. The obvious general criterion for variable selection is to choose the ‘most representative’ or ‘most relevant’ variables. However, it is also clear that a purely relevance-oriented criterion could lead to select many redundant variables. The minimum Redundance Maximum Relevance (mRMR) procedure, proposed by Ding and Peng [Minimum redundancy feature selection from microarray gene expression data. J Bioinform Comput Biol. 2005;3:185–205] and Peng et al. [Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell. 2005;27:1226–1238] is an algorithm to systematically perform variable selection, achieving a reasonable trade-off between relevance and redundancy. In its original form, this procedure is based on the use of the so-called mutual information criterion to assess relevance and redundancy. Keeping the focus on functional data problems, we propose here a modified version of the mRMR method, obtained by replacing the mutual information by the new association measure (called distance correlation) suggested by Székely et al. [Measuring and testing dependence by correlation of distances. Ann Statist. 2007;35:2769–2794]. We have also performed an extensive simulation study, including 1600 functional experiments (100 functional models sample sizes classifiers) and three real-data examples aimed at comparing the different versions of the mRMR methodology. The results are quite conclusive in favour of the new proposed alternative.
1 May 2018 in Statistics & Probability Letters
1 February 2017 in The Journal of Nutritional Biochemistry