Hao Sun

Hao Sun


Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
17 October 2021 in Remote Sensing
Verified
Journal Article
Open access
PDF file
YXYanjie Xu
Yanjie Xu
HSHao Sun
Hao Sun
JCJin Chen
Jin Chen
LLLin Lei
Lin Lei
KJKefeng Ji
Kefeng Ji
GKGangyao Kuang
Gangyao Kuang
Synthetic aperture radar (SAR) can perform observations at all times and has been widely used in the military field. Deep neural network (DNN)-based SAR target recognition models have achieved great success in recent years. Yet, the adversarial robustness of these models has received far less academic attention in the remote sensing community. In this article, we first present a comprehensive adversarial robustness evaluation framework for DNN-based SAR target recognition. Both data-oriented metrics and model-oriented metrics have been used to fully assess the recognition performance under adversarial scenarios. Adversarial training is currently one of the most successful methods to improve the adversarial robustness of DNN models. However, it requires class labels to generate adversarial attacks and suffers significant accuracy dropping on testing data. To address these problems, we introduced adversarial self-supervised learning into SAR target recognition for the first time and proposed a novel unsupervised adversarial contrastive learning-based defense method. Specifically, we utilize a contrastive learning framework to train a robust DNN with unlabeled data, which aims to maximize the similarity of representations between a random augmentation of a SAR image and its unsupervised adversarial example. Extensive experiments on two SAR image datasets demonstrate that defenses based on adversarial self-supervised learning can obtain comparable robust accuracy over state-of-the-art supervised adversarial learning methods.

11 October 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
HWHan Wu
Han Wu
HSHao Sun
Hao Sun
KJKefeng Ji
Kefeng Ji
GKGangyao Kuang
Gangyao Kuang
Multi-drone multi-object tracking (MDMOT) aims to localize and identify targets from videos captured simultaneously by multiple drones. To accomplish this task, existing methods typically follow the strategy of associating localized targets to obtain identities. However, their localization and identification stages heavily rely on single-frame information, resulting in the localization being very sensitive to visual information decay and making it struggle to capture discriminative representations for target identification. Consequently, they usually exhibit unreliable performance in challenging scenarios, such as occlusion and high similarity among targets. To this end, we introduce a novel MDMOT framework to interact temporal-spatial features, exploring the guidance of tracklet information across time and space. Specifically, we introduce temporal-spatial feedback loops to enrich cues in our tracker. Meanwhile, a novel temporal-oriented target localization is proposed to enhance the response to difficult samples in feature space by utilizing prior knowledge from existing tracklets beyond the current frame for target localization. Moreover, a spatial-oriented target identification is designed to synergize cross-drone information of tracklets, thereby providing discriminative representations for target identification. It combines target and background information to extract identity representations and interacts features from multiple drones. To our best knowledge, this work reports the first MDMOT system that synergizes features across multiple drones to track targets. By incorporating these two elaborated networks, we develop a robust tracker (named TSMMT). Extensive experiments on the MDMT public dataset demonstrate the superiority of our proposed model. Specifically, TSMMT outperforms state-of-the-art methods by 2.76%∼4.66% on MOTA and 2.06%∼3.33% on IDF1.

16 September 2022 in Remote Sensing
Verified
Journal Article
Open access
PDF file
Chenfang LiuChenfang Liu
Chenfang Liu
HSHao Sun
Hao Sun
YXYanjie Xu
Yanjie Xu
GKGangyao Kuang
Gangyao Kuang
SAR-optical images from different sensors can provide consistent information for scene classification. However, the utilization of unlabeled SAR-optical images in deep learning-based remote sensing image interpretation remains an open issue. In recent years, contrastive self-supervised learning (CSSL) methods have shown great potential for obtaining meaningful feature representations from massive amounts of unlabeled data. This paper investigates the effectiveness of CSSL-based pretraining models for SAR-optical remote-sensing classification. Firstly, we analyze the contrastive strategies of single-source and multi-source SAR-optical data augmentation under different CSSL architectures. We find that the CSSL framework without explicit negative sample selection naturally fits the multi-source learning problem. Secondly, we find that the registered SAR-optical images can guide the Siamese self-supervised network without negative samples to learn shared features, which is also the reason why the CSSL framework outperforms the CSSL framework with negative samples. Finally, we apply the CSSL pretrained network without negative samples that can learn the shared features of SAR-optical images to the downstream domain adaptation task of optical transfer to SAR images. We find that the choice of a pretrained network is important for downstream tasks.

28 March 2024 in IEEE Transactions on Circuits and Systems for Video Technology
Verified
Journal Article
Closed access
JSJian Sun
Jian Sun
HSHao Sun
Hao Sun
LLLin Lei
Lin Lei
KJKefeng Ji
Kefeng Ji
GKGangyao Kuang
Gangyao Kuang
Cross-view geo-localization aims to associate geographical location with different view images shot from different platforms. One of the critical challenges is how to effectively emphasize architectural features and reducing background interference to achieve robust cross-view matching. Most of the existing methods fail to adequately address the features of buildings, treating foreground and background equally. Leveraging prior knowledge to enhance the features of crucial architectural foreground yields greater benefits in Geo-Localization. A comprehensive three stage approach (TirSA) is proposed in this paper, which consists of three components: Pre-processing, Generate Feature Embedding, and Post-processing. In the Pre-processing stage, we employ a self-supervised feature enhancement method (SFEM) to obtain the building aware mask. Without adding additional auxiliary information, the model is guided to learn from discriminative building regions. Besides, in the Generate Feature Embedding stage, we propose an adaptive feature integration module (AFIM) to enhance feature representation capability. We also train the Siamese network using a novel improved cross-domain triplet loss to reduce the impact of inter-view domain gap. Finally, in the Post-processing stage, we employ a re-ranking method to optimize the initial retrieval list, further enhancing the matching accuracy. Remarkably, extensive experiments show that our proposed TirSA exceeds state-of-the-art by a large margin and achieves optimality in both drone-view target localization and drone navigation. Especially in the drone navigation task, our method is superior to the existing methods, achieving an improvement of approximately 5%. Code will be released at https://github.com/SunJ1025/TirSA.

8 June 2023 in Remote Sensing
Verified
Journal Article
Open access
PDF file
ZLZihao Lu
Zihao Lu
HSHao Sun
Hao Sun
YXYanjie Xu
Yanjie Xu
Deep neural networks (DNNs) have been widely utilized in automatic visual navigation and recognition on modern unmanned aerial vehicles (UAVs), achieving state-of-the-art performances. However, DNN-based visual recognition systems on UAVs show serious vulnerability to adversarial camouflage patterns on targets and well-designed imperceptible perturbations in real-time images, which poses a threat to safety-related applications. Considering a scenario in which a UAV is suffering from adversarial attack, in this paper, we investigate and construct two ensemble approaches with CNN and transformer for both proactive (i.e., generate robust models) and reactive (i.e., adversarial detection) adversarial defense. They are expected to be secure under attack and adapt to the resource-limited environment on UAVs. Specifically, the probability distributions of output layers from base DNN models in the ensemble are combined in the proactive defense, which mainly exploits the weak adversarial transferability between the CNN and transformer. For the reactive defense, we integrate the scoring functions of several adversarial detectors with the hidden features and average the output confidence scores from ResNets and ViTs as a second integration. To verify their effectiveness in the recognition task of remote sensing images, we conduct experiments on both optical and synthetic aperture radar (SAR) datasets. We find that the ensemble model in proactive defense performs as well as three popular counterparts, and both of the ensemble approaches can achieve much more satisfactory results than a single base model/detector, which effectively alleviates adversarial vulnerability without extra re-training. In addition, we establish a one-stop platform for conveniently evaluating adversarial robustness and performing defense on recognition models called AREP-RSIs, which is beneficial for the future research of the remote sensing field.

22 September 2023 in Remote Sensing
Verified
Journal Article
Open access
PDF file
ZLZihao Lu
Zihao Lu
HSHao Sun
Hao Sun
KJKefeng Ji
Kefeng Ji
GKGangyao Kuang
Gangyao Kuang
As a safety-related application, visual systems based on deep neural networks (DNNs) in modern unmanned aerial vehicles (UAVs) show adversarial vulnerability when performing real-time inference. Recently, deep ensembles with various defensive strategies against adversarial samples have drawn much attention due to the increased diversity and reduced variance for their members. Aimed at the recognition task of remote sensing images (RSIs), this paper proposes to use a reactive-proactive ensemble defense framework to solve the security problem. In reactive defense, we fuse scoring functions of several classical detection algorithms with the hidden features and average output confidences from sub-models as a second fusion. In terms of proactive defense, we attempt two strategies, including enhancing the robustness of each sub-model and limiting the transferability among sub-models. In practical applications, the real-time RSIs are first input to the reactive defense part, which can detect and reject the adversarial RSIs. The accepted ones are then passed to robust recognition with a proactive defense. We conduct extensive experiments on three benchmark RSI datasets (i.e., UCM, AID, and FGSC-23). The experimental results show that the deep ensemble method of reactive and proactive defense performs very well in gradient-based attacks. The analysis of the applicable attack scenarios for each proactive ensemble defense is also helpful for this field. We also perform a case study with the whole framework in the black-box scenario, and the highest detection rate reaches 93.25%. Most of the adversarial RSIs can be rejected in advance or correctly recognized by the enhanced deep ensemble. This article is the first one to combine reactive and proactive defenses with a deep ensemble against adversarial attacks in the context of RSI recognition for DNN-based UAVs.