We propose a novel online manifold regularization framework based on the notion of duality in constrained optimization. The Fenchel conjugate of hinge functions is a key to transfer manifold regularization from offline to online in this paper. Our algorithms are derived by gradient ascent in the dual function. For practical purpose, we propose two buffering strategies and two sparse approximations to reduce the computational complexity. Detailed experiments verify the utility of our approaches. An important conclusion is that our online MR algorithms can handle the settings where the target hypothesis is not fixed but drifts with the sequence of examples. We also recap and draw connections to earlier works. This paper paves a way to the design and analysis of online manifold regularization algorithms.

We propose a novel online coregularization framework for multiview semisupervised learning based on the notion of duality in constrained optimization. Using the weak duality theorem, we reduce the online coregularization to the task of increasing the dual function. We demonstrate that the existing online coregularization algorithms in previous work can be viewed as an approximation of our dual ascending process using gradient ascent. New algorithms are derived based on the idea of ascending the dual function more aggressively. For practical purpose, we also propose two sparse approximation approaches for kernel representation to reduce the computational complexity. Experiments show that our derived online coregularization algorithms achieve risk and accuracy comparable to offline algorithms while consuming less time and memory. Specially, our online coregularization algorithms are able to deal with concept drift and maintain a much smaller error rate. This paper paves a way to the design and analysis of online coregularization algorithms.

Online semi-supervised learning (OS2L) has received much attention recently because of its well practical usefulness. Most of existing studies of online semi-supervised learning related to manifold regularization. In this study, we propose a novel online semi-supervised learning framework with multiple regularization terms based on the notion of ascending the dual function in constrained optimization. Using Fenchel conjugate, different semi-supervised regularization terms can be integrated in the dual function easily and directly. This approach is derived by updating limited dual coefficient variables on each learning round. To be practical, we also employ buffering strategies and sparse approximation approaches in this work. Experimental studies show that our methods achieve accuracy comparable to offline algorithms while consuming less time and memory. Specially, our online semi-supervised learning algorithms can handle the settings where the target hyperplane of classification continually drifts with the sequence of arriving instances. This paper paves a way to the design and analysis of online semi-supervised learning algorithms with multiple regularization terms.

In the fields of machine learning and artificial intelligence, the processing of time-series data has been a continuous concern and a significant algorithm for intelligent applications. Traditional deep-learning-based methods seem to have reached performance ceilings in certain specific areas, such as online character recognition. This paper proposes an algorithmic framework to break this deadlock by classifying time-series data by evaluating the similarities among handwriting samples using multidimensional Dynamic Time Warping (mDTW) distances. A simplified hierarchical clustering algorithm is employed as a classifier for character recognition. Moreover, this work achieves joint modeling with current mainstream temporal models, enabling the mDTW model to integrate modeling results from methods like RNN or Transformer, therefore further enhancing the accuracy of related algorithms. A series of experiments were conducted on a public database, and the results indicate that our method overcomes the bottleneck of current deep-learning-based methods in the field of online handwriting character recognition. More importantly, compared to deep -learning-based methods, the proposed method has a simpler structure and higher interpretability. Experimental results demonstrate that our proposed method outperforms existing state-of-the-art models in handwriting character recognition, achieving a top-1 accuracy of 98.5% and a top-3 accuracy of 99.3%, thus confirming its effectiveness in overcoming the limitations of traditional deep-learning models in temporal sequence processing.