By assuming latent sources are statistically independent, independent component analysis separates underlying sources from a given linear mixture. Since in many applications, latent sources are both non-Gaussian and have sample dependence, it is desirable to exploit both properties jointly. In this paper, we use mutual information rate to construct a general framework for analysis and derivation of algorithms that take both properties into account. We discuss two types of source models for entropy rate estimation—a Markovian and an invertible filter model—and give the general independent component analysis cost function, update rule, and performance analysis based on these. We also introduce four algorithms based on these two models, and show that their performance can approach the Cramér-Rao lower bound. In addition, we demonstrate that the algorithms with flexible models exhibit very desirable performance for “natural” data.

By assuming latent sources are statistically independent, independent component analysis (ICA) separates underlying sources from a given linear mixture. Since in many applications, latent sources are non-Gaussian, noncircular, and have sample dependence, it is desirable to exploit all these properties jointly. Mutual information rate, which leads to the minimization of entropy rate, provides a natural cost for the task. In this paper, we establish the theory for complex-valued ICA giving Cramér-Rao lower bound and identification conditions, and present a new algorithm that takes all these properties into account. We propose an effective estimator of entropy rate and a complex-valued entropy rate bound minimization algorithm based on it. We show that the new method exploits all these properties effectively by comparing the estimation performance with the Cramér-Rao lower bound and by a number of examples.