This paper addresses the issue of estimating from a given data sequence the number of mixture components for a Gaussian mixture model(GMM). Our approach is to compute the normalized maximum likelihood (NML) code length for the data sequence relative to a GMM, then to find the mixture size that attains the minimum of the NML on the basis of the minimum description length principle. For finite domains, Kontkanen and Myllymäki proposed a method for efficient computation of the NML code length for specific models, however, for general classes over infinite domains, it has remained open how we compute the NML code length efficiently. We first propose a general method for calculating the NML code length for a general exponential family. Then, we apply it to the efficient computation of the NML code length for a GMM. The key idea is to restrict the data domain in combination with the technique of employing a generating function for computing the normalization term for a GMM. We use artificial datasets to empirically demonstrate that our estimate of the mixture size converges to the true one significantly faster than other criteria.

This paper corrects errors in the calculation of the normalized maximum likelihood (NML) code length for a Gaussian mixture model (GMM). It shows that the NML code length calculated in “Efficient computation of normalized maximum likelihood codes for Gaussian mixture models with its applications to clustering” is an upper bound on the NML code length strictly calculated for the Gaussian Mixture Model. Additionally, we correct the NML code length for generalized logistic distribution.

This paper addresses the issues of how we can quantify structural information for nonparametric distributions and how we can detect its changes. Structural information refers to an index for a global understanding of a data distribution. When we consider the problem of clustering using a parametric model such as a Gaussian mixture model, the number of mixture components (clusters) can be thought of as structural information in the model. However, there does not exist any notion of structural information for nonparametric modeling of data. In this paper we introduce a novel notion of {\em kernel complexity} (KC) as structural information in the nonparametric setting. The key idea of KC is to combine the information bias inspired by the Gini index with the information quantity measured in terms of the normalized maximum likelihood (NML) code length. We empirically show that KC has a property similar to the number of clusters in a parametric model. We further propose a framework for structural change detection with KC in nonparametric distributions. With synthetic and real data sets we empirically demonstrate that our framework enables us to detect structural changes underlying the data and their early warning signals.