In convolutional neural networks (CNNs), higher layer information is more abstract and more task specific, so people usually concern themselves with fully connected (FC) layer features, believing that lower layer features are less discriminative. However, a few researchers showed that the lower layers also provide very rich and powerful information for image representation. In view of these study findings, in this letter, we attempt to adaptively and explicitly combine the activations from intermediate and FC layers to generate a new CNN with directed acyclic graph topology, which is called the converted CNN. After that, two converted CNNs are integrated together to further improve the classification performance. We validate our proposed two-stage deep feature fusion model over two publicly available remote sensing data sets, and achieve a state-of-the-art performance in scene classification tasks.

Scene classification is a fundamental task for automatic remote sensing image understanding. In recent years, convolutional neural networks have become a hot research topic in the remote sensing community, and have made great achievements in scene classification. Deep convolutional networks are primarily trained in a supervised way, requiring huge volumes of labeled training samples. However, clearly labeled remote sensing data are usually limited. To address this issue, in this paper, we propose a novel scene classification method via triplet networks, which use weakly labeled images as network inputs. Besides, we initiate a theoretical study on the three existing loss functions for triplet networks, analyzing their different underlying mechanisms for dealing with “hard” and/or “easy” triplets during training. Furthermore, four new loss functions are constructed, aiming at laying more stress on “hard” triplets to improve classification accuracy. Extensive experiments have been conducted, and the experimental results show that triplet networks coupled with our proposed losses achieve a state-of-the-art performance in scene classification tasks.

In the field of content-based remote sensing (RS) image retrieval, convolutional neural networks (CNNs) have been demonstrating overwhelming superiority among other methods in terms of performance. CNNs are basically trained in a supervised way, requiring a larger number of labeled samples. However, scarcity of labeled images is very prevalent in the RS community. Moreover, CNN-based approaches have other disadvantages such as cumbersome networks and high-dimensional features. To address these issues, we apply unsupervised transfer learning to CNN training--we transform similarity learning into deep ordinal classification with the help of several CNN experts pretrained over large-scale-labeled everyday image sets, which jointly determine image similarities and provide pseudolabels for classification. Our proposed method ends up with a brand-new lightweight model called similarity-based Siamese CNN (SBS-CNN), which can be trained from scratch with completely unlabeled RS images, and whose resulting features are compact. Furthermore, the existing CNNs are generally coupled with the cross-entropy loss, entirely ignoring the interclass semantic relationship. To overcome this shortcoming, a novel loss function called weighted Wasserstein ordinal loss is constructed to take into account the ordinal relationship among categories, thus more effectively navigating parameter updates during training. Extensive experiments have been carried out over publicly available RS data sets, and it turns out that our SBS-CNN outperforms existing CNN-based approaches.

In multiclass classification, convolutional neural network (CNN) is generally coupled with the cross-entropy (CE) loss, which only penalizes the predicted probability corresponding to a ground truth class and ignores the interclass relationship. We argue that CNN can be improved by using a better loss function. On the other hand, the Wasserstein distance (WD) is a well-known metric used to measure the distance between two distributions. Directly solving the WD problem requires a prohibitively large amount of computation time, whereas the cheaper iterative algorithms have a variety of shortcomings such as computational instability and difficulty in selecting parameters. In this paper, we address these issues by giving an analytical solution to the WD problem--for the first time, we find that for two distributions in hierarchically organized data space, WD has a closed-form solution, which we call ``hierarchical WD (HWD).'' We use this theory to construct novel loss functions that overcome the shortcomings of CE loss. To this end, multi-CNN information fusion that provides the basis for building category hierarchies is carried out first. Then, the semantic relationship among classes is modeled as a binary tree. Then, CNN coupled with an HWD-based loss, i.e., hierarchical Wasserstein CNN (HW-CNN), is trained to learn deep features. In this way, prior knowledge about the interclass relationship is embedded into HW-CNN, and information from several CNNs provides guidance in the process of training individual HW-CNNs. We conducted extensive experiments over two publicly available remote sensing data sets and achieved a state-of-the-art performance in scene classification tasks.

In recent years, convolutional neural networks (CNNs) have become the predominant method for content-based aerial image retrieval (CBAIR) and aerial scene classification (ASC) due to their overwhelming performance advantages. However, existing CNN-based models have the following shortcomings: first, they do not deal with large intraclass variations, thereby overlooking the possibility of fine-grained retrieval and classification; second, all similarity learning methods for CBAIR consider similarity between two images as a constant, neglecting the fact that image similarity is uncertain in nature; third, similarity learning is separated from ASC, ignoring the advantages of joint optimization. To address these issues, we propose a novel metric learning method called center-metric learning, and couple it with a new kind of loss called positive-negative center loss, which, with the help of several “experts,” enables CNNs to cope successfully with within-class variations. Besides, we propose similarity distribution learning, making the first attempt to embed uncertainty regarding similarity into the training process. The resulting fine-grained similarity predictions can further strengthen CNNs' fine discrimination ability. Furthermore, three tasks, that is, center-metric learning, similarity distribution learning, and ASC, are incorporated into one CNN, benefitting from one another and leading to a better generalization capability. Just like an eagle, our model is able to discriminate subtle differences among aerial images, hence the name “eagle-eyed multitask CNN.” We carry out extensive experiments over four publicly available aerial image sets and achieve a performance better than all existing methods.

A careful assessment of contextual factors and human behavior is essential when implementing population health strategies, such as dietary salt reduction to reduce the intake of excessive sodium, which is the top-ranked dietary risk factor associated with cardiovascular diseases.^1,2 A significant number of strokes are caused by high blood pressure due to overconsumption of sodium.^3,4 There has been substantial evidence from randomized clinical trials showing that reduced sodium intake leads to a decrease in blood pressure.⁵ However, definitive evidence on whether sodium reduction can reduce incident cardiovascular disease events is lacking, to our knowledge.