Robots are coming to help us in different harsh environments such as deep sea or coal mine. Waste landfill is the place like these with casualty risk, gas poisoning, and explosion hazards. It is reasonable to use robots to fulfill tasks like burying operation, transportation, and inspection. In these assignments, one important issue is to obtain appropriate paths for robots especially in some complex applications. In this context, a novel hybrid swarm intelligence algorithm, ant colony optimization enhanced by chaos-based particle swarm optimization, is proposed in this article to deal with the path planning problem for landfill inspection robots in Asahikawa, Japan. In chaos-based particle swarm optimization, Chebyshev chaotic sequence is used to generate the random factors for particle swarm optimization updating formula so as to effectively adjust particle swarm optimization parameters. This improved model is applied to optimize and determine the hyper parameters for ant colony optimization. In addition, an improved pheromone updating strategy which combines the global asynchronous feature and “Elitist Strategy” is employed in ant colony optimization in order to use global information more appropriately. Therefore, the iteration number of ant colony optimization invoked by chaos-based particle swarm optimization can be reduced reasonably so as to decrease the search time effectively. Comparative simulation experiments show that the chaos-based particle swarm optimization-ant colony optimization has a rapid search speed and can obtain solutions with similar qualities.

Multi-label image classification is more in line with the real-world applications. This problem is difficult due to the the fact that complex label space makes it hard to get label-level attention regions and deal with semantic relationships among labels. Common deep network-based methods utilize CNN to extract features and consider the labels as a sequence or a graph, thus handling the label correlations with RNN or graph-theoretical algorithms. In this paper, we propose a novel CNN-RNN-based model, bi-modal multi-label learning(BMML) framework. Firstly, an improved channel-wise attention mechanism is presented to propose regional attention maps and connect them to relative labels. After that, based on the assumption that objects in a semantic scene always have high-level relevance among visual and textual corpus, we further embed the labels through different pre-trained language models and determine the label sequence in a “semantic space” constructed on large-scale textual data, thereby handling the labels in their semantic context. In addition, a cross-modal feature aligning module is introduced in BMML framework. Experimental results show that BMML is able to achieve better accuracies then those mainstream multi-label classification methods on several benchmark data sets.

A deep understanding of our visual world is more than an isolated perception on a series of objects, and the relationships between them also contain rich semantic information. Especially for those satellite remote sensing images, the span is so large that the various objects are always of different sizes and complex spatial compositions. Therefore, the recognition of semantic relations is conducive to strengthen the understanding of remote sensing scenes. In this paper, we propose a novel multi-scale semantic fusion network (MSFN). In this framework, dilated convolution is introduced into a graph convolutional network (GCN) based on an attentional mechanism to fuse and refine multi-scale semantic context, which is crucial to strengthen the cognitive ability of our model Besides, based on the mapping between visual features and semantic embeddings, we design a sparse relationship extraction module to remove meaningless connections among entities and improve the efficiency of scene graph generation. Meanwhile, to further promote the research of scene understanding in remote sensing field, this paper also proposes a remote sensing scene graph dataset (RSSGD). We carry out extensive experiments and the results show that our model significantly outperforms previous methods on scene graph generation. In addition, RSSGD effectively bridges the huge semantic gap between low-level perception and high-level cognition of remote sensing images.

The results of aerial scene classification can provide valuable information for urban planning and land monitoring. In this specific field, there are always a number of object-level semantic classes in big remote-sensing pictures. Complex label-space makes it hard to detect all the targets and perceive corresponding semantics in the typical scene, thereby weakening the sensing ability. Even worse, the preparation of a labeled dataset for the training of deep networks is more difficult due to multiple labels. In order to mine object-level visual features and make good use of label dependency, we propose a novel framework in this article, namely a Cross-Modal Representation Learning and Label Graph Mining-based Residual Multi-Attentional CNN-LSTM framework (CM-GM framework). In this framework, a residual multi-attentional convolutional neural network is developed to extract object-level image features. Moreover, semantic labels are embedded by language model and then form a label graph which can be further mapped by advanced graph convolutional networks (GCN). With these cross-modal feature representations (image, graph and text), object-level visual features will be enhanced and aligned to GCN-based label embeddings. After that, aligned visual signals are fed into a bi-LSTM subnetwork according to the built label graph. The CM-GM framework is able to map both visual features and graph-based label representations into a correlated space appropriately, using label dependency efficiently, thus improving the LSTM predictor’s ability. Experimental results show that the proposed CM-GM framework is able to achieve higher accuracy on many multi-label benchmark datasets in remote sensing field.

Unmanned aerial vehicle is a typical field robot which can work in many unstructured environments like mines, forests, and even radiation areas. In our mine monitoring system built in a northeast province of China, special designed unmanned aerial vehicle is applied to take photos and perceive the environment. We select a series of image-based techniques to process aerial pictures to monitor the slope. The visual features are initially refined by histogram equalization. Then, the rocks and cracks can be detected by different digital image processing operators, like Canny, so as to assess displacements. Advanced semantic segmentation model, U-Net, is also selected to process the problem. Experimental results show that both Canny and U-Net can perceive the edges in pictures effectively, better than other operators. In addition, we model the inspection mission for mine slopes into a traveling salesman problem, then plan the path for unmanned aerial vehicle by swarm intelligence-based optimization.

Fine-grained image categorization is still a challenging computer vision problem in recent years. Most of existing methods highly rely on massive labeled data which are scarce in many real world applications. It should also be noticed that progressive learning demands of existing data is very common today. That is, we may pay attention to more fine-grained information (like arctic tern, black tern, buttercup or tulip) in an existing data set with labels like “bird” and “flower”. It is reasonable to believe that the existing labels and model with transferable knowledge would be helpful to another related but different, fine-grained recognition task. In this context, an improved transfer deep learning approach with hierarchical multi-adversarial networks is proposed in this paper. With this approach, cross domain features are extracted by advanced deep encoders coarsely. After that, we annotate a small amount of images in the target domain, thereby creating the “active labels” which can provide instructions for adversarial learning. Then, the GAN-based hierarchical model is utilized to select cross domain categories and enhance related features so as to facilitate an effective transfer. In order to exploit useful local features, a novel adaptive attention mechanism, Region Adversarial Network (RAN) which can select attention regions during adversarial learning and generate valuable fine-grained features, is introduced in the article. We call the proposed hierarchical framework “Attentional Multi-Adversarial Networks (AMAN)” . Experimental results show that AMAN is able to augment cross domain features well-directly and build an effective classifier for fine-grained categorization in the target domain with fewer training samples and higher accuracies.