Hao Jin

Dr. Hao Jin

orcid

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 March 2025 in IEEE Transactions on Geoscience and Remote Sensing
Verified
Journal Article
Closed access
HLHui Lin
Hui Lin
Danfeng HongDanfeng Hong
Danfeng Hong
SGShuhang Ge
Shuhang Ge
Chuyao LuoChuyao Luo
Chuyao Luo
KJKai Jiang
Kai Jiang
CWCongcong Wen
Congcong Wen
Remote Sensing Image Captioning (RSIC) presents unique challenges and plays a critical role in applications such as environmental monitoring, urban planning, and disaster management. Traditional RSIC methods often struggle to produce rich and diverse descriptions. Recently, with significant advancements in Vision-Language Models (VLMs), efforts have emerged to integrate these models into the remote sensing domain and to introduce richly descriptive datasets specifically designed to enhance VLM training. However, most current RSIC models generally apply only fine-tuning to these datasets without developing models tailored to the unique characteristics of remote sensing imagery. This paper proposes RS-MoE, the first Mixture of Expert based VLM specifically customized for remote sensing domain. Unlike traditional MoE models, the core of RS-MoE is the MoE Block, which incorporates a novel Instruction Router and multiple lightweight Large Language Models (LLMs) as expert models. The Instruction Router is designed to generate specific prompts tailored for each corresponding LLM, guiding them to focus on distinct aspects of the RSIC task. This design not only allows each expert LLM to concentrate on a specific subset of the task, thereby enhancing the specificity and accuracy of the generated captions, but also improves the scalability of the model by facilitating parallel processing of sub-tasks. Additionally, we present a two-stage training strategy for tuning our RS-MoE model to prevent performance degradation due to sparsity. We fine-tuned our model on the RSICap dataset using our proposed training strategy. Experimental results on the RSICap dataset, along with evaluations on other traditional datasets where no additional fine-tuning was applied, demonstrate that our model achieves state-of-the-art performance in generating precise and contextually relevant captions. Notably, our RS-MoE-1B variant achieves performance comparable to 13B VLMs, demonstrating the efficiency of our model design. Moreover, our model demonstrates promising generalization capabilities by consistently achieving state-of-the-art performance on the Remote Sensing Visual Question Answering (RSVQA) task.

16 April 2024 in IEEE Transactions on Knowledge and Data Engineering
Verified
Journal Article
Closed access
JLJun Li
Jun Li
Bin YiBin Yi
Bin Yi
LPLiang Peng
Liang Peng
YYYang Yang
Yang Yang
Yangyang LiYangyang Li
Yangyang Li
ZHZi Huang
Zi Huang
In the absence of an official statement about a rumor, people may expose the truth behind such rumor through their responses on social media. Due to the varying relevance of responses in exposing hidden suspicious points within a rumor claim, it is crucial to prioritize those with higher relevance, rather than considering every responding tweets. As for the multi-modal rumor detection, an effective approach for evaluating relevance is aligning responses with the different modalities of the rumor claim in a fine-grained manner. However, owing to the substantial volume of response tweets, it is both costly and redundant to align all responses with the multi-modal claim. In this paper, we propose a novel two-stage model, termed Focal Reasoning Model (FoRM) , to select critical responses for multi-modal rumor detection. More specifically, our FoRM consists of two primary elements: coarse-grained selection and fine-grained reasoning. The coarse-grained selection component employs post-level features of responses to initialize a relevant score for each. Based on these scores, we preserve the responses with higher scores as the candidate ones for subsequent reasoning. Within the fine-grained reasoning component, we develop a relation attention module to investigate fine-grained relationships, specifically token-to-token and token-to-object connections, between the preserved responses and the multi-modal claim, with the goal of discovering valuable clues. Extensive experiments have been conducted on three real-world datasets, and the results demonstrate that our proposed model outperforms all the baselines.

10 July 2023 in Electronics
Verified
Journal Article
Open access
PDF file
XLXiaohan Liu
Xiaohan Liu
YZYue Zhan
Yue Zhan
HJHao Jin
Hao Jin
YWYuan Wang
Yuan Wang
YZYi Zhang
Yi Zhang
In order to ensure the healthy development of social networks and the harmony and stability of the society, as well as to facilitate effective supervision by regulatory authorities, a classification method of social bots is proposed based on the identification of social bots in the early stage. First of all, the topic-related introduction is used to expand the topic, and on this basis, the SBERT (Sentence-BERT) model is applied to make relevance judgments between the micro-blog text and the expanded topics to identify polluters. Then, an opinion sentence recognition method that combines social bots opinion sentence generation rules with a deep learning model TextCNN is proposed to further distinguish commenters and spreaders. Finally, in order to improve the classification effect of the model, the transfer learning method is used to train the model with the help of a large number of micro-blogs of ordinary Weibo accounts, so as to better improve the classification effect of social bots. The comparative experimental results show that the topic expansion method can effectively improve the classification results of the SBERT model for the relevance of micro-blog text topics. When the parameter k of the expanded topic model is set at 20, the content of the expanded topic sequence is more consistent with the core content of most Weibo text sequences, and the obtained model has the best performance. By analyzing the opinion-based micro-blog text generation rules of social bots, focusing on the keywords that express opinions, the problem of difficulty in recognizing opinion sentences produced by the low quality of opinion sentences of social bots is well resolved, and the recognition effect of opinion sentences has been improved by more than 10%. Through the introduction of transfer learning, the problem of insufficient social bots data is effectively alleviated, and the classification effect of social bots is greatly improved, with an increase of more than 10%.


19 December 2024 in IEEE Geoscience and Remote Sensing Letters
Verified
Journal Article
Closed access
HLHui Lin
Hui Lin
Ying ZhangYing Zhang
Ying Zhang
JZJizhe Zhou
Jizhe Zhou
XWXuejiao Wang
Xuejiao Wang
HJHao Jin
Hao Jin
CWCongcong Wen
Congcong Wen
In this work, we analyze outgoing longwave radiation (OLR) data from the National Oceanic and Atmospheric Administration to extract TIR anomalies over mainland China from 2007 to 2019. These anomalies are statistically significantly correlated with earthquakes of magnitude ≥ 4.5, 5.0 and 5.5. We then calculate the distance ( d /°) and time lag ( t / days ) between each TIR anomaly and the corresponding earthquake. By examining the distribution of earthquakes in t × d space, our results show that the normalized frequency of earthquakes as a function of distance and time follows a power law distribution: f ( d , t , M ) = ϕ( M ) * t −dt(M) * d −dd(M) , where f ( d , t , M ) represents the frequency of earthquakes with magnitude ≥ M occurring within the time-space unit ( t , d ) around TIR anomalies. The coefficient ϕ( M ) is negatively correlated with magnitude [ϕ(4.5) = 0.31, ϕ(5.0) = 0.30, ϕ(5.5) = 0.28], while dd ( M ) and dt ( M ) are two positive coefficients [ dt (4.5) = 1.59, dt (5.0) = 1.58, dt (5.5) = 1.50; dd (4.5) = 2.76, dd (5.0) = 2.66, dd (5.5) = 2.59]. A larger ϕ suggests that earthquakes are more spatially and temporally concentrated around TIR anomalies, while dd and dt represent the decay rates of frequency with distance and time, respectively. These findings imply that earthquakes are more likely to occur near TIR anomalies, and the model f ( d , t , M ) could potentially be used to construct a probabilistic earthquake forecasting system based on TIR anomalies.

21 May 2025 in IEEE Transactions on Geoscience and Remote Sensing
Verified
Journal Article
Closed access
WCWeiyi Chen
Weiyi Chen
JGJinchao Gui
Jinchao Gui
YLYuchen Li
Yuchen Li
HJHao Jin
Hao Jin
LZLisha ZHOU
Lisha ZHOU
QKQin Kai
Qin Kai
Visual image-based localization is important to intelligent systems, greatly influencing Earth science uses. However, variability in lighting conditions can impair the performance of such systems, particularly for single-image localization tasks. This article proposes an original all-day visual localization model that incorporates visible and infrared (IR) imagery, boosting its robustness to lighting changes. The proposed model takes on a large-scale architecture, exploiting the capabilities of both image modalities to achieve better localization accuracy across different lighting situations. The contributions of this study include: 1) a multimodal fusion approach correlates visual data with geographic coordinates, refining the precision of image localization; 2) an adaptive fusion mechanism is implemented, adjusting the contribution of each image stream in response to environmental cues, thereby ensuring more reliable localization outcomes; and 3) a lightweight localization method that utilizes a single image for outdoor localization, eliminating the need for additional sensor data. We have conducted extensive testing, including shared datasets and random experiments, and the results show that our designed model not only improves localization accuracy in different dimensions compared to other models but also enhances the coverage of the lighting environment. It can also achieve image localization in nighttime environments and simulate the ability of human eyes to perceive environmental location information.