Since about 100 years ago, to learn the intrinsic structure of data, many representation learning approaches have been proposed, either linear or nonlinear, either supervised or unsupervised, either “shallow” or “deep”. Particularly, deep architectures are widely applied for representation learning in recent years, and have delivered top results in many tasks, such as image classification, object detection and speech recognition. In this paper, we review the development of data representation learning methods. Specifically, we investigate both traditional feature learning algorithms and state-of-the-art deep learning models. The history of data representation learning is introduced, while available online resources (e.g., courses, tutorials and books) and toolboxes are provided. At the end, we give a few remarks on the development of data representation learning and suggest some interesting research directions in this area

Since Pearson developed principal component analysis (PCA) in 1901, feature learning (or called representation learning) has been studied for more than 100 years. During this period, many “shallow” feature learning methods have been proposed based on various learning criteria and techniques, until the popular deep learning research in recent years. In this advanced review, we describe the historical profile of the shallow feature learning research and introduce the important developments of the deep learning models. Particularly, we survey the deep architectures with benefits from the optimization of their width and depth, as these models have achieved new records in many applications, such as image classification and object detection. Finally, several interesting directions of deep learning are presented and briefly discussed.This article is categorized under: Technologies > Classification Technologies > Machine Learning Technologies > Computational Intelligence

The original random forests (RFs) algorithm has been widely used and has achieved excellent performance for the classification and regression tasks. However, the research on the theory of RFs lags far behind its applications. In this article, to narrow the gap between the applications and the theory of RFs, we propose a new RFs algorithm, called random Shapley forests (RSFs), based on the Shapley value. The Shapley value is one of the well-known solutions in the cooperative game, which can fairly assess the power of each player in a game. In the construction of RSFs, RSFs use the Shapley value to evaluate the importance of each feature at each tree node by computing the dependency among the possible feature coalitions. In particular, inspired by the existing consistency theory, we have proved the consistency of the proposed RFs algorithm. Moreover, to verify the effectiveness of the proposed algorithm, experiments on eight UCI benchmark datasets and four real-world datasets have been conducted. The results show that RSFs perform better than or at least comparable with the existing consistent RFs, the original RFs, and a classic classifier, support vector machines.

Deep learning, a data-driven technology, has attracted widespread attention from various disciplines due to the rapid advancements in the Internet of Things (IoT) big data, machine learning algorithms and computational hardware in recent years. It proves to achieve comparable or even more accurate results than traditional methods in a more flexible manner in existing applications in various fields. In the field of physical oceanography, an important scientific field of oceanography, the abundance of ocean surface data and high dynamic complexity pave the way for an extensive application of deep learning. Moreover, researchers have already conducted a great deal of work to innovate traditional approaches in ocean circulation, ocean dynamics, ocean climate, ocean remote sensing and ocean geophysics, leading oceanographic studies into the “AI ocean era”. In our study, we categorize numerous research topics in physical oceanography into four aspects: surface elements, subsurface elements, typical ocean phenomena, and typical weather and climate phenomena. We review the cutting-edge applications of deep learning in physical oceanography over the past three years to provide comprehensive insights into its development. From the perspective of three application scenarios, namely spatial data, temporal data and data generation, three corresponding deep learning model types are introduced, which are convolutional neural networks (CNNs), recurrent neural networks (RNNs) and generative adversarial networks (GANs), and also their principal application tasks. Furthermore, this study discusses the current bottlenecks and future innovative prospects of deep learning in oceanography. Through summarizing and analyzing the existing research, our aim is to delve into the potential and challenges of deep learning in physical oceanography, providing reference and inspiration for researchers in future oceanographic studies.