In the age of artificial intelligence, the best approach to handling huge amounts of data is a tremendously motivating and hard problem. Among machine learning models, stochastic gradient descent (SGD) is not only simple but also very effective. This study provides a detailed analysis of contemporary state-of-the-art deep learning applications, such as natural language processing (NLP), visual data processing, and voice and audio processing. Following that, this study introduces several versions of SGD and its variant, which are already in the PyTorch optimizer, including SGD, Adagrad, adadelta, RMSprop, Adam, AdamW, and so on. Finally, we propose theoretical conditions under which these methods are applicable and discover that there is still a gap between theoretical conditions under which the algorithms converge and practical applications, and how to bridge this gap is a question for the future.

Multiview learning (MVL), by exploiting the complementary information among multiple feature sets, can improve the performance of many existing learning tasks. Support vector machine (SVM)-based models have been frequently used for MVL. A typical SVM-based MVL model is SVM-2K, which extends SVM for MVL by using the distance minimization version of kernel canonical correlation analysis. However, SVM-2K cannot fully unleash the power of the complementary information among different feature views. Recently, a framework of learning using privileged information (LUPI) has been proposed to model data with complementary information. Motivated by LUPI, we propose a new multiview privileged SVM model, multi-view privileged SVM model (PSVM-2V), for MVL. This brings a new perspective that extends LUPI to MVL. The optimization of PSVM-2V can be solved by the classical quadratic programming solver. We theoretically analyze the performance of PSVM-2V from the viewpoints of the consensus principle, the generalization error bound, and the SVM-2K learning model. Experimental results on 95 binary data sets demonstrate the effectiveness of the proposed method.