1 December 2020 in Pattern Recognition
10 February 2020 in IEEE Transactions on Multimedia
Cross-modal analysis has become a promising direction for artificial intelligence. Visual representation is crucial for various cross-modal analysis tasks that require visual content understanding. Visual features which contain semantical information can disentangle the underlying correlation between different modalities, thus benefiting the downstream tasks. In this paper, we propose a Visual Reasoning and Attention Network (VRANet) as a plug-and-play module to capture rich visual semantics and help to enhance the visual representation for improving cross-modal analysis. Our proposed VRANet is built based on the bilinear visual attention module which identifies the critical objects. We propose a novel Visual Relational Reasoning (VRR) module to reason about pair-wise and inner-group visual relationships among objects guided by the textual information. The two modules enhance the visual features at both relation level and object level. We demonstrate the effectiveness of the proposed VRANet by applying it to both Visual Question Answering (VQA) and Cross-Modal Information Retrieval (CMIR) tasks. Extensive experiments conducted on VQA 2.0, CLEVR, CMPlaces, and MS-COCO datasets indicate superior performance comparing with state-of-the-art work.
1 August 2021 in Information Fusion
5 November 2020 in IEEE Access
Cognitive radio network is an intelligent wireless communication system which can adjust its transmission parameters according to the environment thanks to its learning ability. It is a feasible and promising direction to solve the spectrum scarcity issue and has become a research focus in communication community. However, cognitive radio network is vulnerable to jamming attack, resulting in serious degradation of spectrum utilization. In this paper, we view the anti-jamming task of cognitive radio as a Markov decision process and propose an intelligent anti-jamming scheme based on deep reinforcement learning. We aim to learn a policy for users to maximize their rate of successful transmission. Specifically, we design Double Deep Q Network (Double DQN) to model the confrontation between the cognitive radio network and the jammer. The Q network is implemented using Transformer encoder to effectively estimate action-values from raw spectrum data. The simulation results indicate that our approach can effectively defend against several kinds of jamming attacks.
2 August 2022 in IEEE Internet of Things Journal
In recent years, with the rise of intelligent hardware technology, the Internet of Things (IoT) has achieved rapid development and has been widely used in smart cities, smart power grids, industrial Internet, and other fields. The resulting security risks of IoT have attracted more and more attention. Being different from the traditional authentication that is based on MAC or security certificate, RF fingerprinting technology extracts fingerprints from the emissions of wireless transmitters. These fingerprints root in the hardware imperfection of the transmitting circuits and can be used for wireless device identification and authentication. RF fingerprinting can effectively enhance the security of the wireless network and has become a research focus in the field of IoT. However, how to learn and fuse multiple RF fingerprints is still an urgent problem to be solved. Inspired by the attention mechanism in the field of computer vision, an adaptive RF fingerprints fusion network (ARFNet) is proposed in this article, which is built on our dual attention convolution (DAConv) layer. This neural network can extract and adaptively fuse multiple RF fingerprints in a data-driven manner to obtain more discriminant features. In addition, a data augmentation method is also designed to effectively enhance the network’s robustness to channel variation and SNR variation. Extensive experimental results show that the proposed ARFNet combined with data augmentation can achieve 99.5% recognition accuracy on five USRP X310, and achieve 95.7% recognition accuracy on 56 ADS-B devices. Our source code has been released at https://github.com/zhangweifeng1218/Adaptive_RF_Fingerprinting .
3 November 2020 in IEEE Transactions on Image Processing
Different from Visual Question Answering task that requires to answer only one question about an image, Visual Dialogue task involves multiple rounds of dialogues which cover a broad range of visual content that could be related to any objects, relationships or high-level semantics. Thus one of the key challenges in Visual Dialogue task is to learn a more comprehensive and semantic-rich image representation that can adaptively attend to the visual content referred by variant questions. In this paper, we first propose a novel scheme to depict an image from both visual and semantic views. Specifically, the visual view aims to capture the appearance-level information in an image, including objects and their visual relationships, while the semantic view enables the agent to understand high-level visual semantics from the whole image to the local regions. Furthermore, on top of such dual-view image representations, we propose a Dual Encoding Visual Dialogue (DualVD) module, which is able to adaptively select question-relevant information from the visual and semantic views in a hierarchical mode. To demonstrate the effectiveness of DualVD, we propose two novel visual dialogue models by applying it to the Late Fusion framework and Memory Network framework. The proposed models achieve state-of-theart results on three benchmark datasets. A critical advantage of the DualVD module lies in its interpretability. We can analyze which modality (visual or semantic) has more contribution in answering the current question by explicitly visualizing the gate values. It gives us insights in understanding of information selection mode in the Visual Dialogue task. The code is available at https://github.com/JXZe/Learning_DualVD.