Machine learning develops rapidly, which has made many theoretical breakthroughs and is widely applied in various fields. Optimization, as an important part of machine learning, has attracted much attention of researchers. With the exponential growth of data amount and the increase of model complexity, optimization methods in machine learning face more and more challenges. A lot of work on solving optimization problems or improving optimization methods in machine learning has been proposed successively. The systematic retrospect and summary of the optimization methods from the perspective of machine learning are of great significance, which can offer guidance for both developments of optimization and machine learning research. In this article, we first describe the optimization problems in machine learning. Then, we introduce the principles and progresses of commonly used optimization methods. Finally, we explore and give some challenges and open problems for the optimization in machine learning.

Clustering is a machine learning paradigm of dividing sample subjects into a number of groups such that subjects in the same groups are more similar to those in other groups. With advances in information acquisition technologies, samples can frequently be viewed from different angles or in different modalities, generating multiview data. Multiview clustering (MVC), that clusters subjects into subgroups using multiview data, has attracted more and more attentions. Although MVC methods have been developed rapidly, there has not been enough survey to summarize and analyze the current progress. Therefore, we propose a novel taxonomy of the MVC approaches. Similar to other machine learning methods, we categorize them into generative and discriminative classes. In the discriminative class, based on the way of view integration, we split it further into five groups—common eigenvector matrix, common coefficient matrix, common indicator matrix, direct combination, and combination after projection. Furthermore, we relate MVC to other topics: multiview representation, ensemble clustering, multitask clustering, multiview supervised, and semisupervised learning. Several representative real-world applications are elaborated for practitioners. Some benchmark multiview datasets are introduced and representative MVC algorithms from each group are empirically evaluated to analyze how they perform on benchmark datasets. To promote future development of MVC approaches, we point out several open problems that may require further investigation and thorough examination. Impact Statement—Multiview clustering has gained the success in a variety of applications in the past decade. In order to obtain a comprehensive picture of the MVC development, we provide a new categorization of existing MVC methods and introduce the representative algorithms in each category. At last, we point out open problems that are worth investigating to advance the MVC study. More promising MVC methods to solve these open problems may appear following this review paper from which a large number of applications can benefit.

Traffic flow forecasting, especially the short-term case, is an important topic in intelligent transportation systems (ITS). This paper researches network-scale modeling and forecasting of short-term traffic flows. First, the concepts of single-link and multilink models of traffic flow forecasting are proposed. Secondly, four prediction models are constructed by combining the two models with single-task learning (STL) and multitask learning (MTL). The combination of the multilink model and multitask learning not only improves the experimental efficiency but also improves the prediction accuracy. Moreover, a new multilink, single-task approach that combines graphical lasso (GL) with neural network (NN) is proposed. GL provides a general methodology for solving problems involving lots of variables. Using L1 regularization, GL builds a sparse graphical model, making use of the sparse inverse covariance matrix. Gaussian process regression (GPR) is a classic regression algorithm in Bayesian machine lea...

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods.