This paper investigates Kalman Filter-based Heuristic Ensemble (KFHE), which is a new perspective on multi-class ensemble classification with performance significantly better or at least as good as the state-of-the-art algorithms. We prove that the sample weight tuning method used in KFHE is a version of adaptive boosting, and the weight distribution does not change anymore and leads to redundant classifiers when the algorithm iterates enough times. This motivates us to select a sub-ensemble to alleviate the redundancy and improve the performance of the ensemble. An Ordering-based Kalman Filter Selective Ensemble (OKFSE) is proposed in this paper to select a sub-ensemble using the margin distance minimization approach. We demonstrate the effectiveness and robustness of OKFSE through extensive experiments on 20 real-world UCI datasets, and the statistical test shows that OKFSE significantly outperforms the state-of-the-art KFHE and clustering-based pruning methods on these datasets with 5% and 10% class label noise.

Subspace clustering aims to find clusters in the low-dimensional subspaces for high-dimensional data. Subspace clustering with Block Diagonal Representation (BDR) maintains the number of connected components of the graph by Laplacian rank constraint, and the learned affinity matrix shows a block diagonal structure, which will achieve a good segmentation for the dataset by spectral clustering. However, the subspaces of real data may overlap and the learned affinity matrix may be imprecise. In this work, we propose an Active learning framework for BDR(ABDR) to acquire and incorporate prior knowledge to improve the subspace clustering performance. An active selection strategy is designed to acquire labels of the informative data points from both the skeleton of clusters and the boundaries of clusters, and then the labeled data are converted into pairwise constraints, which are incorporated into BDR. The optimization of the new objective function is given and the convergence of ABDR is discussed. Experimental results on three images datasets(MNIST, ORL and COIL-20) and one UCI dataset(ISOLET) demonstrate the effectiveness of ABDR on complex clustering tasks and show that ABDR is superior to multiple state-of-the-art active clustering and learning techniques.

As a relatively advanced method, the subspace clustering algorithm by block diagonal representation (BDR) will be competent in performing subspace clustering on a dataset if the dataset is assumed to be noise-free and drawn from the union of independent linear subspaces. Unfortunately, this assumption is far from reality, since the real data are usually corrupted by various noises and the subspaces of data overlap with each other, the performance of linear subspace clustering algorithms, including BDR, degrades on the real complex data. To solve this problem, we design a new objective function based on BDR, in which l_2,1 norm of the reconstruction error is introduced to model the noises and improve the robustness of the algorithm. After optimizing the objective function, we present the corresponding subspace clustering algorithm to pursue a self-expressive coefficient matrix with a block diagonal structure for a noisy dataset. An affinity matrix is constructed based on the coefficient matrix, and then fed to the spectral clustering algorithm to obtain the final clustering results. Experiments on several artificial noisy image datasets show that the proposed algorithm has robustness and better clustering performance than the compared algorithms.