This page has only limited features, please log in for full access.

Unclaimed
Kun Fu
Laboratory of Technology in Geo-spatial Information Processing and Application System, Chinese Academy of Sciences Aerospace Information Research Institute, 560203 Beijing, Beijing, China

Basic Info

Basic Info is private.

Honors and Awards

The user has no records in this section


Career Timeline

The user has no records in this section.


Short Biography

The user biography is not available.
Following
Followers
Co Authors
The list of users this user is following is empty.
Following: 0 users

Feed

Journal article
Published: 24 August 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Reads 0
Downloads 0

In this paper, we introduce the 2020 Gaofen Challenge and relevant scientific outcomes. The 2020 Gaofen Challenge is an international competition, which is organized by the China High-Resolution Earth Observation Conference Committee and the Aerospace Information Research Institute, Chinese Academy of Sciences and technically co-sponsored by the IEEE Geoscience and Remote Sensing Society (IEEE-GRSS) and the International Society for Photogrammetry and Remote Sensing (ISPRS). It aims at promoting the academic development of automated high-resolution earth observation image interpretation. Six independent tracks have been organized in this challenge, which cover the challenging problems in the field of object detection and semantic segmentation. With the development of convolutional neural networks, deep learning-based methods have achieved good performance on image interpretation. In this paper, we report the details and the best-performing methods presented so far in the scope of this challenge.

ACS Style

Xian Sun; Peijin Wang; Zhiyuan Yan; Wenhui Diao; Xiaonan Lu; Zhujun Yang; Yidan Zhang; Deliang Xiang; Chen Yan; Jie Guo; Bo Dang; Wei Wei; Feng Xu; Cheng Wang; Ronny Hansch; Martin Weinmann; Naoto Yokoya; Kun Fu. Automated High-resolution Earth Observation Image Interpretation: Outcome of the 2020 Gaofen Challenge. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2021, PP, 1 -1.

AMA Style

Xian Sun, Peijin Wang, Zhiyuan Yan, Wenhui Diao, Xiaonan Lu, Zhujun Yang, Yidan Zhang, Deliang Xiang, Chen Yan, Jie Guo, Bo Dang, Wei Wei, Feng Xu, Cheng Wang, Ronny Hansch, Martin Weinmann, Naoto Yokoya, Kun Fu. Automated High-resolution Earth Observation Image Interpretation: Outcome of the 2020 Gaofen Challenge. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2021; PP (99):1-1.

Chicago/Turabian Style

Xian Sun; Peijin Wang; Zhiyuan Yan; Wenhui Diao; Xiaonan Lu; Zhujun Yang; Yidan Zhang; Deliang Xiang; Chen Yan; Jie Guo; Bo Dang; Wei Wei; Feng Xu; Cheng Wang; Ronny Hansch; Martin Weinmann; Naoto Yokoya; Kun Fu. 2021. "Automated High-resolution Earth Observation Image Interpretation: Outcome of the 2020 Gaofen Challenge." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing PP, no. 99: 1-1.

Journal article
Published: 13 August 2021 in IEEE Transactions on Geoscience and Remote Sensing
Reads 0
Downloads 0

With the proposal of neural architecture search (NAS), automated network architecture design gradually becomes a new way in deep learning research. Due to its high capability regarding automated design, some pioneers have made an attempt to apply NAS in remote sensing and made some achievements, like 1-D/3-D Auto-convolutional neural network (CNN) and polarimetric synthetic aperture radar (PolSAR)-tailored Differentiable Architecture Search (PDAS). However, there are still some areas to be improved for existing NAS in remote-sensing field. In this article, we propose a random topology and random multiscale mapping (RTRMM) method to generate a multiscale and lightweight architecture for remote-sensing image recognition. First, a random topology generator generates the topology through random graph. Second, during the experiment, we find remote-sensing image features extracted by a multiscale network are more appropriate, compared with features extracted by a single-scale model. Nevertheless, the complexity inevitably increases with the introduction of a multiscale concept. Consequently, we design a variable search space consisting of decomposition convolution units under the guidance of mathematical analysis. The mapping of each neuron is then determined by a random multiscale mapping sampler. After that, we assemble the topology and mappings into blocks and construct three RTRMM models. Experiments on four scene classification datasets confirm the feature extraction capability and lightweight performance of RTRMM models. Moreover, we also observe that our approach achieves a better tradeoff between floating-point operations (FLOPs) and accuracy than some current well-behaved methods. Furthermore, the results on Vaihingen dataset verify the high feature-transfer capability.

ACS Style

Jihao Li; Martin Weinmann; Xian Sun; Wenhui Diao; Yingchao Feng; Kun Fu. Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition. IEEE Transactions on Geoscience and Remote Sensing 2021, PP, 1 -17.

AMA Style

Jihao Li, Martin Weinmann, Xian Sun, Wenhui Diao, Yingchao Feng, Kun Fu. Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition. IEEE Transactions on Geoscience and Remote Sensing. 2021; PP (99):1-17.

Chicago/Turabian Style

Jihao Li; Martin Weinmann; Xian Sun; Wenhui Diao; Yingchao Feng; Kun Fu. 2021. "Random Topology and Random Multiscale Mapping: An Automated Design of Multiscale and Lightweight Neural Network for Remote-Sensing Image Recognition." IEEE Transactions on Geoscience and Remote Sensing PP, no. 99: 1-17.

Journal article
Published: 01 June 2021 in Neurocomputing
Reads 0
Downloads 0

Knowledge graph embedding, as the upstream task of link prediction which aims to predict new links between entities under the premise of known relations, its reliability greatly affects the performance of link prediction. However, previous distance-based models focus on modeling complicated relation patterns while ignoring the semantic hierarchy of knowledge graph, from TransE to RotatE. In this setting, all entities are regarded as the same type, and the fact that different entities belong to different levels is neglected. Therefore, we propose the general form of RotatE, the hierarchical-aware relation rotational knowl- edge graph embedding (HA-RotatE), to model the hierarchical-aware knowledge graph. HA-RotatE represents entities and relations as complex vectors and uses different moduli of entity embeddings to indicate the different hierarchical levels they belong to. The transformation of modulus and rotation from head entity to tail entity depends on different relations. Some relations are used to link entities of the same level, and others are used to link entities of different levels. We adopt the shared modulus transformation parameter method for avoiding overfitting. As the general form of RotatE, HA-RotatE also has the ability to model and infer various relation modes, i.e., symmetry/antisymmetric, inversion and composition. On benchmark datasets WN18RR and FB15k-237, the experiments on link prediction tasks show that: (1) HA-RotatE can effectively model the semantic hierarchy of the knowledge graph; (2) Compared with competitive benchmarks, our model substantially outperforms them in most metrics.

ACS Style

Shensi Wang; Kun Fu; Xian Sun; Zequn Zhang; Shuchao Li; Li Jin. Hierarchical-aware relation rotational knowledge graph embedding for link prediction. Neurocomputing 2021, 458, 259 -270.

AMA Style

Shensi Wang, Kun Fu, Xian Sun, Zequn Zhang, Shuchao Li, Li Jin. Hierarchical-aware relation rotational knowledge graph embedding for link prediction. Neurocomputing. 2021; 458 ():259-270.

Chicago/Turabian Style

Shensi Wang; Kun Fu; Xian Sun; Zequn Zhang; Shuchao Li; Li Jin. 2021. "Hierarchical-aware relation rotational knowledge graph embedding for link prediction." Neurocomputing 458, no. : 259-270.

Journal article
Published: 10 May 2021 in IEEE Transactions on Geoscience and Remote Sensing
Reads 0
Downloads 0

With the rapid update and iteration of current aerial image data, the continual learning scenarios and catastrophic forgetting problem attracted increased attention, especially in the semantic segmentation task. However, the existing methods mainly focus on the class continual learning in a single task and are not satisfactory when extended to multiple tasks. In this article, we consider more realistic and complicated settings, namely task continual learning. We revisit the characteristics of semantic segmentation and knowledge distillation (KD) strategy, then propose a general and effective framework, named structured inheritance, to learn new tasks while retaining high performance on old tasks. Specifically, we present two structure-preserving penalties: pixel affinity structure loss and representation consistency structure loss. The former breaks the isolation of pixels and retains the pixel interactive information learned by the old tasks. At the same time, the latter protects high-frequency stationary information between sequence semantic segmentation tasks. Our approach does not need to add extra parameters nor does it need to access the data stream of the old tasks. Therefore, it can be applied in practical applications with strict computational burden, memory cost, and storage budget. Extensive continual learning experiments on four semantic segmentation datasets of Vaihingen, Potsdam, DeepGlobe, and Gaofen challenge semantic segmentation dataset (GCSS) prove the effectiveness of our proposed framework, which outperforms the current state-of-the-art methods and even exceeds the theoretical upper-bound performance of multitask learning. The code and models will be made publicly available.

ACS Style

Yingchao Feng; Xian Sun; Wenhui Diao; Jihao Li; Xin Gao; Kun Fu. Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery. IEEE Transactions on Geoscience and Remote Sensing 2021, PP, 1 -17.

AMA Style

Yingchao Feng, Xian Sun, Wenhui Diao, Jihao Li, Xin Gao, Kun Fu. Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery. IEEE Transactions on Geoscience and Remote Sensing. 2021; PP (99):1-17.

Chicago/Turabian Style

Yingchao Feng; Xian Sun; Wenhui Diao; Jihao Li; Xin Gao; Kun Fu. 2021. "Continual Learning With Structured Inheritance for Semantic Segmentation in Aerial Imagery." IEEE Transactions on Geoscience and Remote Sensing PP, no. 99: 1-17.

Technical note
Published: 29 April 2021 in Remote Sensing
Reads 0
Downloads 0

This paper proposed a new remote sensing observation capability evaluation model (RSOCE) based on analytic hierarchy process to quantitatively evaluate the capability of multi-satellite cooperative remote sensing observation. The analytic hierarchical process model is a combination of qualitative and quantitative analysis of systematic decision analysis method. According to the objective of the remote sensing cooperative observation mission, we decompose the complex problem into several levels and a number of factors, compare and calculate various factors in pairs, and obtain the combination weights of different schemes. The model can be used to evaluate the observation capability of resource satellites. Taking the optical remote sensing satellites, such as China’s resource satellite series and GF-4, as examples, this paper verifies and evaluates the model for three typical tasks: point target observation, regional target observation, and moving target continuous observation. The results show that the model can provide quantitative reference and model support for comprehensive evaluation of the collaborative observation capability of remote sensing satellites.

ACS Style

Zhonggang Zheng; Qingmei Li; Kun Fu. Evaluation Model of Remote Sensing Satellites Cooperative Observation Capability. Remote Sensing 2021, 13, 1717 .

AMA Style

Zhonggang Zheng, Qingmei Li, Kun Fu. Evaluation Model of Remote Sensing Satellites Cooperative Observation Capability. Remote Sensing. 2021; 13 (9):1717.

Chicago/Turabian Style

Zhonggang Zheng; Qingmei Li; Kun Fu. 2021. "Evaluation Model of Remote Sensing Satellites Cooperative Observation Capability." Remote Sensing 13, no. 9: 1717.

Journal article
Published: 27 April 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Reads 0
Downloads 0

The building extraction from SAR images has always been a challenging research topic. Recently, the deep convolution neural network brings excellent improvements in SAR segmentation. The FCN and other variants are widely transferred to the SAR studies because of their high precision in optical images. They are still limited by their processing in terms of the geometric distortion of buildings, the variability of building structures, and scattering interference between adjacent targets in the SAR images. In this paper, a unified framework called selective spatial pyramid dilated (SSPD) network is proposed. Firstly, we design the multi-layer SSPD based on the selective attention. The multi-scale building information with different attention on multiple branches is combined, optimized, and adaptively selected for adaptive filtering and extracting features of complex multi-scale building targets in SAR images. Secondly, we propose a novel encoder-decoder structure for the fine building feature reconstruction. The enhanced encoder and the dual-stage decoder, composed of the context balancing module and SSPD, extract and recover the crucial multi-scale information better. Thirdly, according to the building features and SAR imaging mechanism, a new loss function called L-shape weighting loss is proposed to heighten the attention on the L-shape footprint characteristics of the buildings and reduce the missing detection of line buildings. Besides, it can also alleviate the class imbalance problem in the optimization stage. Finally, the experiments on a large-scene SAR image dataset demonstrate the effectiveness of the proposed method and verify its superiority over other approaches, such as the MRF, U-net, and DeepLabv3+.

ACS Style

Hao Jing; Xian Sun; Zhirui Wang; Kaiqiang Chen; Wenhui Diao; Kun Fu. Fine Building Segmentation in High-Resolution SAR Images via Selective Pyramid Dilated Network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2021, PP, 1 -1.

AMA Style

Hao Jing, Xian Sun, Zhirui Wang, Kaiqiang Chen, Wenhui Diao, Kun Fu. Fine Building Segmentation in High-Resolution SAR Images via Selective Pyramid Dilated Network. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2021; PP (99):1-1.

Chicago/Turabian Style

Hao Jing; Xian Sun; Zhirui Wang; Kaiqiang Chen; Wenhui Diao; Kun Fu. 2021. "Fine Building Segmentation in High-Resolution SAR Images via Selective Pyramid Dilated Network." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing PP, no. 99: 1-1.

Journal article
Published: 26 March 2021 in Pattern Recognition
Reads 0
Downloads 0

Weakly-supervised semantic segmentation, as a promising solution to alleviate the burden of collecting per-pixel annotations, aims to train a segmentation model from partial weak annotations. Scribble on the object is one of the commonly used weak annotations and has shown to be sufficient for learning a decent segmentation model. Despite being effective, scribble-based weakly-supervised learning methods often lead to imprecise segmentation on object boundaries. This is mainly because the scribble annotations usually locate inside the objects and the dataset lacks annotations close to the semantic boundaries. To alleviate this issue, this paper proposes a simple-but-effective solution, i.e., BoundaryMix, which generates pseudo training image-annotation pairs from the original images to supplement the missing semantic boundaries. Specifically, given a prediction of segmentation, we cut off the regions around the estimated boundaries, which are error-prone and replace them with the contents from another image, which in effect creates new samples with less ambiguity around semantic boundaries. With training on scribbles and the on-the-fly generated pseudo annotations, the network acquires better prediction capability around the boundary region and thus improves the overall segmentation performance. By conducting experiments on PASCAL VOC 2012 dataset and POTSDAM dataset with only scribble annotations, we demonstrate the excellent performance of the proposed method and the almost closed gap between scribble-supervised and fully-supervised image segmentation.

ACS Style

Wanxuan Lu; Dong Gong; Kun Fu; Xian Sun; Wenhui Diao; Lingqiao Liu. Boundarymix: Generating pseudo-training images for improving segmentation with scribble annotations. Pattern Recognition 2021, 117, 107924 .

AMA Style

Wanxuan Lu, Dong Gong, Kun Fu, Xian Sun, Wenhui Diao, Lingqiao Liu. Boundarymix: Generating pseudo-training images for improving segmentation with scribble annotations. Pattern Recognition. 2021; 117 ():107924.

Chicago/Turabian Style

Wanxuan Lu; Dong Gong; Kun Fu; Xian Sun; Wenhui Diao; Lingqiao Liu. 2021. "Boundarymix: Generating pseudo-training images for improving segmentation with scribble annotations." Pattern Recognition 117, no. : 107924.

Journal article
Published: 24 March 2021 in IEEE Transactions on Geoscience and Remote Sensing
Reads 0
Downloads 0

Semantic segmentation in very-high-resolution (VHR) aerial images is one of the most challenging tasks in remote sensing image understanding. Most of the current approaches are based on deep convolutional neural networks (DCNNs). However, standard convolution with local receptive fields fails in modeling global dependencies. Prior research works have indicated that attention-based methods can capture long-range dependencies and further reconstruct the feature maps for better representation. Nevertheless, limited by the mere perspective of spatial and channel attention and huge computation complexity of self-attention (SA) mechanism, it is unlikely to model the effective semantic interdependencies between each pixel pair of remote sensing data with complex spectra. In this work, we propose a novel attention-based framework named hybrid multiple attention network (HMANet) to adaptively capture global correlations from the perspective of space, channel, and category in a more effective and efficient manner. Concretely, a class augmented attention (CAA) module embedded with a class channel attention (CCA) module can be used to compute category-based correlation and recalibrate the class-level information. In addition, we introduce a simple yet effective region shuffle attention (RSA) module to reduce feature redundant and improve the efficiency of SA mechanism via regionwise representations. Extensive experimental results on the ISPRS Vaihingen, Potsdam benchmark, and iSAID data set demonstrate the effectiveness and efficiency of our HMANet over other state-of-the-art methods.

ACS Style

Ruigang Niu; Xian Sun; Yu Tian; Wenhui Diao; Kaiqiang Chen; Kun Fu. Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images. IEEE Transactions on Geoscience and Remote Sensing 2021, PP, 1 -18.

AMA Style

Ruigang Niu, Xian Sun, Yu Tian, Wenhui Diao, Kaiqiang Chen, Kun Fu. Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images. IEEE Transactions on Geoscience and Remote Sensing. 2021; PP (99):1-18.

Chicago/Turabian Style

Ruigang Niu; Xian Sun; Yu Tian; Wenhui Diao; Kaiqiang Chen; Kun Fu. 2021. "Hybrid Multiple Attention Network for Semantic Segmentation in Aerial Images." IEEE Transactions on Geoscience and Remote Sensing PP, no. 99: 1-18.

Journal article
Published: 08 March 2021 in Applied Sciences
Reads 0
Downloads 0

Ego-network, which can describe relationships between a focus node (i.e., ego) and its neighbor nodes (i.e., alters), often changes over time. Exploring dynamic ego-networks can help users gain insight into how each ego interacts with and is influenced by the outside world. However, most of the existing methods do not fully consider the multilevel analysis of dynamic ego-networks, resulting in some evolution information at different granularities being ignored. In this paper, we present an interactive visualization system called DyEgoVis which allows users to explore the evolutions of dynamic ego-networks at global, local and individual levels. At the global level, DyEgoVis reduces dynamic ego-networks and their snapshots to 2D points to reveal global patterns such as clusters and outliers. At the local level, DyEgoVis projects all snapshots of the selected dynamic ego-networks onto a 2D space to identify similar or abnormal states. At the individual level, DyEgoVis utilizes a novel layout method to visualize the selected dynamic ego-network so that users can track, compare and analyze changes in the relationships between the ego and alters. Through two case studies on real datasets, we demonstrate the usability and effectiveness of DyEgoVis.

ACS Style

Kun Fu; Tingyun Mao; Yang Wang; Daoyu Lin; Yuanben Zhang; Xian Sun. DyEgoVis: Visual Exploration of Dynamic Ego-Network Evolution. Applied Sciences 2021, 11, 2399 .

AMA Style

Kun Fu, Tingyun Mao, Yang Wang, Daoyu Lin, Yuanben Zhang, Xian Sun. DyEgoVis: Visual Exploration of Dynamic Ego-Network Evolution. Applied Sciences. 2021; 11 (5):2399.

Chicago/Turabian Style

Kun Fu; Tingyun Mao; Yang Wang; Daoyu Lin; Yuanben Zhang; Xian Sun. 2021. "DyEgoVis: Visual Exploration of Dynamic Ego-Network Evolution." Applied Sciences 11, no. 5: 2399.

Journal article
Published: 22 February 2021 in IEEE Transactions on Geoscience and Remote Sensing
Reads 0
Downloads 0

The state-of-the-art deep neural networks have made a great breakthrough in remote sensing image classification. However, the heavy dependence on large-scale data sets limits the application of the deep learning to synthetic aperture radar (SAR) automatic target recognition (ATR) field where the target sample set is generally small. In this work, a metalearning framework named MSAR, consisting of a metalearner and a base-learner, is proposed to solve the sample restriction problem, which can learn a good initialization as well as a proper update strategy. After training, MSAR can implement fast adaptation with a few training images on new tasks. To the best of our knowledge, this is the first study to solve a few-shot SAR target classification via metalearning. In particular, the few-task problem is defined by analyzing the effect of available training classes on the performance of metalearning models. In order to reduce the metalearning difficulties caused by the few-task problem, three transfer-learning methods are employed, which can leverage the prior knowledge from the pretraining phase. Besides, we design a hard task mining method for effective metalearning. Based on the Moving and Stationary Target Acquisition and Recognition (MSTAR) data set, a specialized data set named NIST-SAR is devised to train and evaluate the proposed method. The experiments on NIST-SAR have shown that the proposed method yields better performances with the largest absolute improvements of 1.7% and 2.3% for 1-shot and 5-shot, respectively, over the next best, which indicates that the proposed method is promising and metalearning is a feasible solution for few-shot SAR ATR.

ACS Style

Kun Fu; Tengfei Zhang; Yue Zhang; Zhirui Wang; Xian Sun. Few-Shot SAR Target Classification via Metalearning. IEEE Transactions on Geoscience and Remote Sensing 2021, PP, 1 -14.

AMA Style

Kun Fu, Tengfei Zhang, Yue Zhang, Zhirui Wang, Xian Sun. Few-Shot SAR Target Classification via Metalearning. IEEE Transactions on Geoscience and Remote Sensing. 2021; PP (99):1-14.

Chicago/Turabian Style

Kun Fu; Tengfei Zhang; Yue Zhang; Zhirui Wang; Xian Sun. 2021. "Few-Shot SAR Target Classification via Metalearning." IEEE Transactions on Geoscience and Remote Sensing PP, no. 99: 1-14.

Journal article
Published: 03 February 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Reads 0
Downloads 0

Electromagnetic reflection calculation is a very important part of most SAR target image simulation methods. The electromagnetic reflection models are usually approximate formulas derived under certain conditions. Errors between these models and the actual situation can cause significant differences between simulation images and real images. Therefore, we propose a novel modified SAR target image simulation framework, in which the deep neural network (DNN) is embedded to calculate the electromagnetic reflection, so that the DNN can directly learn and fit the electromagnetic reflection models from real SAR images. First, the calculation of radar signal intensity is separated from the intensity accumulation in multiple reflections. Thus, the approximate calculation formulas of electromagnetic reflection can be replaced with the DNN models. Next, the replaced DNN model is trained to learn the electromagnetic reflection model from real SAR images. Finally, the fitted electromagnetic reflection models is applied to simulate images. In this simulation framework, the imaging model is still the original simulation method based on ray tracing, which ensures the correctness and generalization of the simulation method. Experiments show that the proposed simulation method can improve the quality of the simulation images. When the image is normalized to [0,1], the MSE between the simulated SAR images and the real images of the SLICY target can reach 0.003. The visualization results of the models also show that the fitted reflection coefficient calculation curve and the convolution kernel used for image post-processing are consistent with the laws in the theoretical model and the actual situation.

ACS Style

Shengren Niu; Xiaolan Qiu; Bin Lei; Kun Fu. A SAR Target Image Simulation Method With DNN Embedded to Calculate Electromagnetic Reflection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2021, 14, 2593 -2610.

AMA Style

Shengren Niu, Xiaolan Qiu, Bin Lei, Kun Fu. A SAR Target Image Simulation Method With DNN Embedded to Calculate Electromagnetic Reflection. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2021; 14 (99):2593-2610.

Chicago/Turabian Style

Shengren Niu; Xiaolan Qiu; Bin Lei; Kun Fu. 2021. "A SAR Target Image Simulation Method With DNN Embedded to Calculate Electromagnetic Reflection." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14, no. 99: 2593-2610.

Journal article
Published: 02 February 2021 in Remote Sensing
Reads 0
Downloads 0

Semantic segmentation of multi-modal remote sensing images is an important branch of remote sensing image interpretation. Multi-modal data has been proven to provide rich complementary information to deal with complex scenes. In recent years, semantic segmentation based on deep learning methods has made remarkable achievements. It is common to simply concatenate multi-modal data or use parallel branches to extract multi-modal features separately. However, most existing works ignore the effects of noise and redundant features from different modalities, which may not lead to satisfactory results. On the one hand, existing networks do not learn the complementary information of different modalities and suppress the mutual interference between different modalities, which may lead to a decrease in segmentation accuracy. On the other hand, the introduction of multi-modal data greatly increases the running time of the pixel-level dense prediction. In this work, we propose an efficient C3Net that strikes a balance between speed and accuracy. More specifically, C3Net contains several backbones for extracting features of different modalities. Then, a plug-and-play module is designed to effectively recalibrate and aggregate multi-modal features. In order to reduce the number of model parameters while remaining the model performance, we redesign the semantic contextual extraction module based on the lightweight convolutional groups. Besides, a multi-level knowledge distillation strategy is proposed to improve the performance of the compact model. Experiments on ISPRS Vaihingen dataset demonstrate the superior performance of C3Net with 15× fewer FLOPs than the state-of-the-art baseline network while providing comparable overall accuracy.

ACS Style

Zhiying Cao; Wenhui Diao; Xian Sun; Xiaode Lyu; Menglong Yan; Kun Fu. C3Net: Cross-Modal Feature Recalibrated, Cross-Scale Semantic Aggregated and Compact Network for Semantic Segmentation of Multi-Modal High-Resolution Aerial Images. Remote Sensing 2021, 13, 528 .

AMA Style

Zhiying Cao, Wenhui Diao, Xian Sun, Xiaode Lyu, Menglong Yan, Kun Fu. C3Net: Cross-Modal Feature Recalibrated, Cross-Scale Semantic Aggregated and Compact Network for Semantic Segmentation of Multi-Modal High-Resolution Aerial Images. Remote Sensing. 2021; 13 (3):528.

Chicago/Turabian Style

Zhiying Cao; Wenhui Diao; Xian Sun; Xiaode Lyu; Menglong Yan; Kun Fu. 2021. "C3Net: Cross-Modal Feature Recalibrated, Cross-Scale Semantic Aggregated and Compact Network for Semantic Segmentation of Multi-Modal High-Resolution Aerial Images." Remote Sensing 13, no. 3: 528.

Journal article
Published: 26 January 2021 in IEEE Geoscience and Remote Sensing Letters
Reads 0
Downloads 0

In recent years, the stereo positioning technology based on multiview spaceborne synthetic aperture radar (SAR) images has been widely applied in digital surface model extraction. In this letter, problems of the existing methods based on the range-Doppler (RD) model in a multiview stereo solution are pointed out. A robust stereo positioning solution for multiview spaceborne SAR images based on the RD model is proposed. In the proposed method, the traditional RD model is normalized to reduce the model errors caused by the different scales of the range equation and Doppler equation. A weighting strategy is also proposed to improve the positioning accuracy. This strategy is useful for multiview stereo positioning when using images of different satellites with orbital data of different accuracies. The experiments based on GaoFen-3 and TerraSAR-X satellite data sets validate the effectiveness of the proposed method.

ACS Style

Yitong Luo; Xiaolan Qiu; Qian Dong; Kun Fu. A Robust Stereo Positioning Solution for Multiview Spaceborne SAR Images Based on the Range-Doppler Model. IEEE Geoscience and Remote Sensing Letters 2021, PP, 1 -5.

AMA Style

Yitong Luo, Xiaolan Qiu, Qian Dong, Kun Fu. A Robust Stereo Positioning Solution for Multiview Spaceborne SAR Images Based on the Range-Doppler Model. IEEE Geoscience and Remote Sensing Letters. 2021; PP (99):1-5.

Chicago/Turabian Style

Yitong Luo; Xiaolan Qiu; Qian Dong; Kun Fu. 2021. "A Robust Stereo Positioning Solution for Multiview Spaceborne SAR Images Based on the Range-Doppler Model." IEEE Geoscience and Remote Sensing Letters PP, no. 99: 1-5.

Journal article
Published: 21 January 2021 in Remote Sensing
Reads 0
Downloads 0

We developed an extension of a previously proposed classification scheme that is based upon Freeman–Durden and Cloude–Pottier decompositions of polarimetric Synthetic Aperture Radar (SAR) data, along with a Double-Bounce Eigenvalue Relative Difference (DERD) parameter, and a Random Forest (RF) classifier. The extension was done, firstly, by using dual-copolarization SAR data acquired at shorter wavelengths (C- and X-band, in addition to the previously used L-band) and, secondly, by adding indicators derived from the (polarimetric) Kennaugh elements. The performance of the newly developed classification scheme, herein abbreviated as FCDK-RF, was tested using SAR data of exposed intertidal flats. We demonstrate that the FCDK-RF scheme is capable of distinguishing between different sediment types, namely mud and sand, at high spatial accuracies. Moreover, the classification scheme shows good potential in the detection of bivalve beds on the exposed flats. Our results show that the developed FCDK-RF scheme can be applied for the mapping of sediments and habitats in the Wadden Sea on the German North Sea coast using multi-frequency and multi-polarization SAR from ALOS-2 (L-band), Radarsat-2 (C-band) and TerraSAR-X (X-band).

ACS Style

Wensheng Wang; Martin Gade; Kerstin Stelzer; Jörn Kohlus; Xinyu Zhao; Kun Fu. A Classification Scheme for Sediments and Habitats on Exposed Intertidal Flats with Multi-Frequency Polarimetric SAR. Remote Sensing 2021, 13, 360 .

AMA Style

Wensheng Wang, Martin Gade, Kerstin Stelzer, Jörn Kohlus, Xinyu Zhao, Kun Fu. A Classification Scheme for Sediments and Habitats on Exposed Intertidal Flats with Multi-Frequency Polarimetric SAR. Remote Sensing. 2021; 13 (3):360.

Chicago/Turabian Style

Wensheng Wang; Martin Gade; Kerstin Stelzer; Jörn Kohlus; Xinyu Zhao; Kun Fu. 2021. "A Classification Scheme for Sediments and Habitats on Exposed Intertidal Flats with Multi-Frequency Polarimetric SAR." Remote Sensing 13, no. 3: 360.

Journal article
Published: 19 January 2021 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Reads 0
Downloads 0

The rapid development of deep learning brings effective solutions for remote sensing image interpretation. Training deep neural network models usually require a large number of manually labeled samples. However, there is a limitation to obtain sufficient labeled samples in remote sensing field to satisfy the data requirement. Therefore, it is of great significance to conduct the research on few-shot learning for remote sensing image interpretation. First, this article provides a bibliometric analysis of the existing works for remote sensing interpretation related to few-shot learning. Second, two categories of few-shot learning methods, i.e., the data-augmentation-based and the prior-knowledge-based, are introduced for the interpretation of remote sensing images. Then, three typical remote sensing interpretation applications are listed, including scene classification, semantic segmentation, and object detection, together with the corresponding public datasets and the evaluation criteria. Finally, the research status is summarized, and some possible research directions are provided. This article gives a reference for scholars working on few-shot learning research in the remote sensing field.

ACS Style

Xian Sun; Bing Wang; Zhirui Wang; Hao Li; Hengchao Li; Kun Fu. Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 2021, 14, 2387 -2402.

AMA Style

Xian Sun, Bing Wang, Zhirui Wang, Hao Li, Hengchao Li, Kun Fu. Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing. 2021; 14 ():2387-2402.

Chicago/Turabian Style

Xian Sun; Bing Wang; Zhirui Wang; Hao Li; Hengchao Li; Kun Fu. 2021. "Research Progress on Few-Shot Learning for Remote Sensing Image Interpretation." IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 14, no. : 2387-2402.

Journal article
Published: 16 January 2021 in ISPRS Journal of Photogrammetry and Remote Sensing
Reads 0
Downloads 0

In recent years, deep learning-based algorithms have brought great improvements to rigid object detection. In addition to rigid objects, remote sensing images also contain many complex composite objects, such as sewage treatment plants, golf courses, and airports, which have neither a fixed shape nor a fixed size. In this paper, we validate through experiments that the results of existing methods in detecting composite objects are not satisfying enough. Therefore, we propose a unified part-based convolutional neural network (PBNet), which is specifically designed for composite object detection in remote sensing imagery. PBNet treats a composite object as a group of parts and incorporates part information into context information to improve composite object detection. Correct part information can guide the prediction of a composite object, thus solving the problems caused by various shapes and sizes. To generate accurate part information, we design a part localization module to learn the classification and localization of part points using bounding box annotation only. A context refinement module is designed to generate more discriminative features by aggregating local context information and global context information, which enhances the learning of part information and improve the ability of feature representation. We selected three typical categories of composite objects from a public dataset to conduct experiments to verify the detection performance and generalization ability of our method. Meanwhile, we build a more challenging dataset about a typical kind of complex composite objects, i.e., sewage treatment plants. It refers to the relevant information from authorities and experts. This dataset contains sewage treatment plants in seven cities in the Yangtze valley, covering a wide range of regions. Comprehensive experiments on two datasets show that PBNet surpasses the existing detection algorithms and achieves state-of-the-art accuracy.

ACS Style

Xian Sun; Peijin Wang; Cheng Wang; Yingfei Liu; Kun Fu. PBNet: Part-based convolutional neural network for complex composite object detection in remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing 2021, 173, 50 -65.

AMA Style

Xian Sun, Peijin Wang, Cheng Wang, Yingfei Liu, Kun Fu. PBNet: Part-based convolutional neural network for complex composite object detection in remote sensing imagery. ISPRS Journal of Photogrammetry and Remote Sensing. 2021; 173 ():50-65.

Chicago/Turabian Style

Xian Sun; Peijin Wang; Cheng Wang; Yingfei Liu; Kun Fu. 2021. "PBNet: Part-based convolutional neural network for complex composite object detection in remote sensing imagery." ISPRS Journal of Photogrammetry and Remote Sensing 173, no. : 50-65.

Journal article
Published: 06 January 2021 in IEEE Access
Reads 0
Downloads 0

Open Set Fine-Grained Recognition (OSFGR) aims at distinguishing known fine-grained categories from the data (contain known and unknown categories). The feature distribution of the known fine-grained category usually has the characteristics of small inter-class variance and large intra-class variance. Directly using the traditional Open Set Recognition (OSR) method on OSFGR does not achieve competitive performance. This is mainly due to the fact that the traditional OSR method is designed based on SoftMax function, whose translation invariance for input weakens the representation ability for the known fine-grained category in OSFGR. To settle this problem, we present a unified method based on class activation mapping value (CAMV) for OSFGR, which preserves the original discriminate feature. Besides, most OSR methods are based on the discrimination model with multi-class thresholds. When most known categories need to recognize, it will face the high computational cost and low efficiency. To overcome this problem, we propose the discrimination model with fusion one-class thresholds, which can greatly improve the inference speed without losing the recognition accuracy. Meanwhile, our idea can settle the extreme situation of OSR (only one known category). Our method achieves the state-of-art performance on the fine-grained open set datasets consisting of 4 public fine-grained datasets.

ACS Style

Wei Dai; Wenhui Diao; Xian Sun; Yue Zhang; Liangjin Zhao; Jun Li; Kun Fu. CAMV: Class Activation Mapping Value Towards Open Set Fine-Grained Recognition. IEEE Access 2021, 9, 8167 -8177.

AMA Style

Wei Dai, Wenhui Diao, Xian Sun, Yue Zhang, Liangjin Zhao, Jun Li, Kun Fu. CAMV: Class Activation Mapping Value Towards Open Set Fine-Grained Recognition. IEEE Access. 2021; 9 (99):8167-8177.

Chicago/Turabian Style

Wei Dai; Wenhui Diao; Xian Sun; Yue Zhang; Liangjin Zhao; Jun Li; Kun Fu. 2021. "CAMV: Class Activation Mapping Value Towards Open Set Fine-Grained Recognition." IEEE Access 9, no. 99: 8167-8177.

Journal article
Published: 05 January 2021 in IEEE Transactions on Geoscience and Remote Sensing
Reads 0
Downloads 0

In recent years, deep convolutional neural networks (DCNNs) have made significant progress in cloud detection tasks, and the detection accuracy has been greatly improved. However, most existing CNN-based models have high computational complexity, which limits their practical application, especially for spaceborne optical remote sensing. In addition, most of the methods cannot make adaptive adjustments based on the structural information of the clouds, and blurred boundaries often occur in the detection results. In order to address these problems, this article proposes a lightweight network (DABNet) to achieve high-accuracy detection of complex clouds, not only a clearer boundary but also lower false-alarm rate. Specifically, a deformable context feature pyramid module is proposed to improve the adaptive modeling capability of multiscale features. Besides, a boundary-weighted loss function is designed to direct the network to focus on cloud boundary information and optimize the relevant detection results. The proposed method has been validated on two data sets: the public GF-1 WFV benchmark and our self-built GF-2 cloud detection data set with higher spatial resolution. The experimental results exhibit that DABNet achieves state-of-the-art performance while only using 4.12M parameters and 8.29G multiadds.

ACS Style

Qibin He; Xian Sun; Zhiyuan Yan; Kun Fu. DABNet: Deformable Contextual and Boundary-Weighted Network for Cloud Detection in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing 2021, PP, 1 -16.

AMA Style

Qibin He, Xian Sun, Zhiyuan Yan, Kun Fu. DABNet: Deformable Contextual and Boundary-Weighted Network for Cloud Detection in Remote Sensing Images. IEEE Transactions on Geoscience and Remote Sensing. 2021; PP (99):1-16.

Chicago/Turabian Style

Qibin He; Xian Sun; Zhiyuan Yan; Kun Fu. 2021. "DABNet: Deformable Contextual and Boundary-Weighted Network for Cloud Detection in Remote Sensing Images." IEEE Transactions on Geoscience and Remote Sensing PP, no. 99: 1-16.

Journal article
Published: 21 December 2020 in Remote Sensing
Reads 0
Downloads 0

Fine-grained aircraft type recognition in remote sensing images, aiming to distinguish different types of the same parent category aircraft, is quite a significant task. In recent decades, with the development of deep learning, the solution scheme for this problem has shifted from handcrafted feature design to model architecture design. Although a great progress has been achieved, this paradigm generally needs strong expert knowledge and rich expert experience. It is still an extremely laborious work and the automation level is relatively low. In this paper, inspired by Neural Architecture Search (NAS), we explore a novel differentiable automatic architecture design framework for fine-grained aircraft type recognition in remote sensing images. In our framework, the search process is divided into several phases. Network architecture deepens at each phase while the number of candidate functions gradually decreases. To achieve it, we adopt different pruning strategies. Then, the network architecture is determined through a potentiality judgment after an architecture heating process. This approach can not only search deeper network, but also reduce the computational complexity, especially for relatively large size of remote sensing images. When all differentiable search phases are finished, the searched model called Fine-Grained Aircraft Type Recognition Net (FGATR-Net) is obtained. Compared with previous NAS, ours are more suitable for relatively large and complex remote sensing images. Experiments on Multitype Aircraft Remote Sensing Images (MTARSI) and Aircraft17 validate that FGATR-Net possesses a strong capability of feature extraction and feature representation. Besides, it is also compact enough, i.e., parameter quantity is relatively small. This powerfully indicates the feasibility and effectiveness of the proposed automatic network architecture design method.

ACS Style

Wei Liang; Jihao Li; Wenhui Diao; Xian Sun; Kun Fu; Yirong Wu. FGATR-Net: Automatic Network Architecture Design for Fine-Grained Aircraft Type Recognition in Remote Sensing Images. Remote Sensing 2020, 12, 4187 .

AMA Style

Wei Liang, Jihao Li, Wenhui Diao, Xian Sun, Kun Fu, Yirong Wu. FGATR-Net: Automatic Network Architecture Design for Fine-Grained Aircraft Type Recognition in Remote Sensing Images. Remote Sensing. 2020; 12 (24):4187.

Chicago/Turabian Style

Wei Liang; Jihao Li; Wenhui Diao; Xian Sun; Kun Fu; Yirong Wu. 2020. "FGATR-Net: Automatic Network Architecture Design for Fine-Grained Aircraft Type Recognition in Remote Sensing Images." Remote Sensing 12, no. 24: 4187.

Journal article
Published: 04 November 2020 in Remote Sensing
Reads 0
Downloads 0

Synthetic Aperture Radar (SAR) target classification is an important branch of SAR image interpretation. The deep learning based SAR target classification algorithms have made remarkable achievements. But the acquisition and annotation of SAR target images are time-consuming and laborious, and it is difficult to obtain sufficient training data in many cases. The insufficient training data can make deep learning based models suffering from over-fitting, which will severely limit their wide application in SAR target classification. Motivated by the above problem, this paper employs transfer-learning to transfer the prior knowledge learned from a simulated SAR dataset to a real SAR dataset. To overcome the sample restriction problem caused by the poor feature discriminability for real SAR data. A simple and effective sample spectral regularization method is proposed, which can regularize the singular values of each SAR image feature to improve the feature discriminability. Based on the proposed regularization method, we design a transfer-learning pipeline to leverage the simulated SAR data as well as acquire better feature discriminability. The experimental results indicate that the proposed method is feasible for the sample restriction problem in SAR target classification. Furthermore, the proposed method can improve the classification accuracy when relatively sufficient training data is available, and it can be plugged into any convolutional neural network (CNN) based SAR classification models.

ACS Style

Wei Liang; Tengfei Zhang; Wenhui Diao; Xian Sun; Liangjin Zhao; Kun Fu; Yirong Wu. SAR Target Classification Based on Sample Spectral Regularization. Remote Sensing 2020, 12, 3628 .

AMA Style

Wei Liang, Tengfei Zhang, Wenhui Diao, Xian Sun, Liangjin Zhao, Kun Fu, Yirong Wu. SAR Target Classification Based on Sample Spectral Regularization. Remote Sensing. 2020; 12 (21):3628.

Chicago/Turabian Style

Wei Liang; Tengfei Zhang; Wenhui Diao; Xian Sun; Liangjin Zhao; Kun Fu; Yirong Wu. 2020. "SAR Target Classification Based on Sample Spectral Regularization." Remote Sensing 12, no. 21: 3628.