Zhengyuan Zhang

Dr. Zhengyuan Zhang

orcid

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
31 March 2022 in IEEE Transactions on Geoscience and Remote Sensing
Verified
Journal Article
Closed access
YZYuan Zhiqiang
Yuan Zhiqiang
WZWenkai zhang
Wenkai zhang
CTChangyuan Tian
Changyuan Tian
XRXuee Rong
Xuee Rong
ZZZhengyuan Zhang
Zhengyuan Zhang
XSXian Sun
Xian Sun
Cross-modal remote sensing text-image retrieval (RSCTIR) has recently become an urgent research hotspot due to its ability of enabling fast and flexible information extraction on remote sensing (RS) images. However, current RSCTIR methods mainly focus on global features of RS images, which leads to the neglect of local features that reflect target relationships and saliency. In this article, we first propose a novel RSCTIR framework based on global and local information (GaLR), and design a multi-level information dynamic fusion (MIDF) module to efficaciously integrate features of different levels. MIDF leverages local information to correct global information, utilizes global information to supplement local information, and uses the dynamic addition of the two to generate prominent visual representation. To alleviate the pressure of the redundant targets on the graph convolution network (GCN) and to improve the model’s attention on salient instances during modeling local features, the denoised representation matrix and the enhanced adjacency matrix (DREA) are devised to assist GCN in producing superior local representations. DREA not only filters out redundant features with high similarity, but also obtains more powerful local features by enhancing the features of prominent objects. Finally, to make full use of the information in the similarity matrix during inference, we come up with a plug-and-play multivariate rerank (MR) algorithm. The algorithm utilizes the $k$ nearest neighbors of the retrieval results to perform a reverse search, and improves the performance by combining multiple components of bidirectional retrieval. Extensive experiments on public datasets strongly demonstrate the state-of-the-art performance of GaLR methods on the RSCTIR task. The code of GaLR method, MR algorithm, and corresponding files have been made available at: https://github.com/xiaoyuan1996/GaLR .

1 December 2021 in IEEE Transactions on Geoscience and Remote Sensing
Verified
Journal Article
Closed access
ZZZhengyuan Zhang
Zhengyuan Zhang
WZWenkai zhang
Wenkai zhang
MYMenglong Yan
Menglong Yan
XGXin Gao
Xin Gao
KFKun Fu
Kun Fu
XSXian Sun
Xian Sun
The encoder–decoder framework is prevalent in existing remote-sensing image captioning (RSIC) models. The appearance of attention mechanisms brings significant results. However, current attention-based caption models only build up the relationships between the local features without introducing the global visual feature and removing redundant feature components. It will cause caption models to generate descriptive sentences that are weakly related to the scene of images. To solve the problems, this article proposed a global visual feature-guided attention (GVFGA) mechanism. First, GVFGA introduces the global visual feature and fuses them with local visual features to build up their relationships between them. Second, an attention gate utilizing the global visual feature is proposed in GVFGA to filter out redundant feature components in the fused image features and provide more salient image features. In addition, to relieve the hidden state’s burden, a linguistic state (LS) is proposed to specifically provide textual features, making the hidden state only guiding visual–textual attention process. What’s more, to further refine the fusion of visual features and textual features, a LS-Guided Attention (LSGA) mechanism is proposed. It can also filter out the irrelevant information in the fused visual–textual feature with the help of an attention gate. The experimental results show that this proposed image captioning model can achieve better results on three RSIC datasets, UCM-Captions, Sydney-Captions, and RSICD datasets.

10 October 2019 in Remote Sensing
Verified
Journal Article
Open access
PDF file
ZZZhengyuan Zhang
Zhengyuan Zhang
WDWenhui Diao
Wenhui Diao
WZWenkai zhang
Wenkai zhang
MYMenglong Yan
Menglong Yan
XGXin Gao
Xin Gao
XSXian Sun
Xian Sun
Significant progress has been made in remote sensing image captioning by encoder-decoder frameworks. The conventional attention mechanism is prevalent in this task but still has some drawbacks. The conventional attention mechanism only uses visual information about the remote sensing images without considering using the label information to guide the calculation of attention masks. To this end, a novel attention mechanism, namely Label-Attention Mechanism (LAM), is proposed in this paper. LAM additionally utilizes the label information of high-resolution remote sensing images to generate natural sentences to describe the given images. It is worth noting that, instead of high-level image features, the predicted categories’ word embedding vectors are adopted to guide the calculation of attention masks. Representing the content of images in the form of word embedding vectors can filter out redundant image features. In addition, it can also preserve pure and useful information for generating complete sentences. The experimental results from UCM-Captions, Sydney-Captions and RSICD demonstrate that LAM can improve the model’s performance for describing high-resolution remote sensing images and obtain better S m scores compared with other methods. S m score is a hybrid scoring method derived from the AI Challenge 2017 scoring method. In addition, the validity of LAM is verified by the experiment of using true labels.

23 February 2022 in IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing
Verified
Journal Article
Open access
YWYong Wang
Yong Wang
WZWenkai zhang
Wenkai zhang
ZZZhengyuan Zhang
Zhengyuan Zhang
XGXin Gao
Xin Gao
XSXian Sun
Xian Sun
Much of the recent work in remote sensing image captioning is influenced by natural image captioning.These methods tend to fix the defects of the model architecture to improve the previous work, but pay little attention to the differences between remote sensing images and natural images. By considering these differences, we propose a multi-scale multi-interaction remote sensing image captioning model. As in Figure~\ref{fig:intro} (a), the targets in remote sensing images have a wide range of scales; while the natural images are generally taken close-up, resulting in a similar scale for the foreground targets. Due to the difference in shooting methods, the model pre-trained on close-up natural images cannot capture multi-scale remote sensing targets well. To alleviate this problem, we propose a two-stage multi-scale structure for feature representation, where we first finetune the CNN backbone on remote sensing images for domain adaption, then we collect features from different stages as the multi-scale feature representation. Moreover, due to the shooting distance, the height information of the target in the remote sensing image is greatly weakened, thus some objects like low plants and grasses become difficult to identify, as in Figure~\ref{fig:intro} (b). Thus, we further propose a multi-interaction feature representation module, where information flow of the same and different layers could effectively interact. By calculating the similarity score among features, we fuse features with high similarity, and increase the distance between features of different categories, thereby enhancing the distinguishability. Results on RSICD, Sydney-Captions and UCM-Captions show a clear improvement over the compared methods.

18 September 2019 in IEEE Access
Verified
Journal Article
Open access
ZZZhengyuan Zhang
Zhengyuan Zhang
WZWenkai Zhang
Wenkai Zhang
WDWenhui Diao
Wenhui Diao
MYMenglong Yan
Menglong Yan
XGXin Gao
Xin Gao
XSXian Sun
Xian Sun
Owing to the effectiveness in selectively focusing on regions of interest of images, the attention mechanism has been widely used in image caption task, which can provide more accurate image information for training deep sequential models. Existing attention-based models typically rely on top-down attention mechanism. While somewhat effective, attention masks in these attention-based models are queried from image features by hidden states of LSTM, rather than optimized by the objective functions. This indirectly supervised training approach cannot ensure that attention layers accurately focus on regions of interest. To address the above issue, in this paper, a novel attention model, Visual Aligning Attention model (VAA), is proposed. In this model, the attention layer is optimized by a well-designed visual aligning loss during the training phase. The visual aligning loss is obtained by explicitly calculating the feature similarity of attended image features and corresponding word embedding vectors. Besides, in order to eliminate the influence of non-visual words in training the attention layer, a visual vocab used for filtering out non-visual words in sentences is proposed, which can neglect the non-visual words when calculating the visual aligning loss. Experiments on UCM-Captions and Sydney-Captions prove that the proposed method is more effective in remote sensing image caption task.