Human action recognition in videos is an important task with a broad range of applications. In this study, we improve the performance of recurrent attention convolutional neural network (RACNN) by proposing a novel model, “attention-again”. We consider the nature of video frames as sequences, which will cause the change of regions of interest in the frame, thus we cannot use an attention mechanism similar to that in images. “Attention-again” model is a variant from traditional attention model for recognizing human activities and is embedded in two long short-term memory (LSTM) layers. Different from hierarchal LSTM which change the LSTM structure to combine the hidden states from two LSTM layers, our proposals introduce “attention-again” model to avoid the change of LSTM structure. Furthermore, this model not only learns the relations in each frame, but also obtains the relations among all frames, and these relations instruct the next learning stage. Therefore, our proposed model outperform the baseline and is superior to methods with the same experimental conditions on three benchmark datasets: UCF-11, HMDB-51 and UCF-101. To understand how the model works, we also visualize the region of interest in the frame.
25 April 2025 in Information Processing & Management
With the continuous development of intelligent UAV technology, efficient and accurate sensing of surrounding objects through onboard sensors has become an important research direction. Among these, object detection is one of the most common perception techniques, but generic object detection methods have low detection performance in remote sensing images. To address this, we propose the weighted learnable recursive aggregation detection framework, which aims to improve the detection performance in UAV remote sensing images. The network maintains efficiency while ensuring the accuracy of detection. Firstly, to improve the fusion capability of multi-scale characterization of small objects, we design a multi-channel weights learnable recursive aggregation network. The network improves the multi-scale representation fusion capability by dynamically fusing different layers of scale features while recursively aggregating different layers of features. In addition, we design a multi-channel recursive residual fusion mechanism for this framework, which is capable of extracting object feature information at different spatial scales and enhances the ability of multi-scale characterization of the object. Then, we migrate the multi-channel weights learnable recursive aggregation network to different detection frameworks to verify its generalization. Finally, we perform experimental validation using VisDrone 2019, AI-TOD dataset and compare it with existing methods. The proposed model improves the mAP@0.5 value by 2.1% over the benchmark model on the VisDrone 2019 dataset. On the AI-TOD dataset, our proposed algorithm improves the mAP@0.5 by 1.7% compared to the benchmark model. Also working on the LLVIP dataset, our algorithm achieves 89.8%.
The rapid development of generative technologies has made the production of forged products easier, and AI-generated forged images are increasingly difficult to accurately detect, posing serious privacy risks and cognitive obstacles to individuals and society. Therefore, constructing an effective method that can accurately detect and locate forged regions has become an important task. This paper proposes a hierarchical and progressive forged image detection and localization method called HPUNet. This method assigns more reasonable hierarchical multi-level labels to the dataset as supervisory information at different levels, following cognitive laws. Secondly, multiple types of features are extracted from AI-generated images for detection and localization, and the detection and localization results are combined to enhance the task-relevant features. Subsequently, HPUNet expands the obtained image features into four different resolutions and performs detection and localization at different levels in a coarse-to-fine cognitive order. To address the limited feature field of view caused by inconsistent forgery sizes, we employ three sets of densely cross-connected hierarchical networks for sufficient interaction between feature images at different resolutions. Finally, a UNet network with a soft-threshold-constrained feature enhancement module is used to achieve detection and localization at different scales, and the reliance on a progressive mechanism establishes relationships between different branches. We use ACC and F1 as evaluation metrics, and extensive experiments on our method and the baseline methods demonstrate the effectiveness of our approach.
29 November 2024 in IEEE Transactions on Instrumentation and Measurement
Camouflaged object detection (COD) is a challenging task that aims to identify objects that blend into their backgrounds. This has various applications in computer vision, including military surveillance, animal behavior analysis, and image editing. However, existing COD methods often face issues of ambiguous boundaries and inaccurate localization due to ineffective feature fusion and boundary guidance. To address these challenges, we propose a new hierarchical synergistic prediction framework, named Detail Perception Synergistic Network (DPSNet), which leverages an incremental strategy and a novel detail perception loss for accurate COD. Specifically, our DPSNet consists of a baseline network and an enhanced network, generating preliminary and fine-grained predictions, respectively. To adapt to the morphological variations of camouflaged objects, we design a Morphological Awareness Adaptation Module (MAAM) that adaptively fuses inconsistent morphological and texture features. To enhance the contextual semantic feature representations and obtain clear predictions, we propose a Semantic Aggregation Enhancement Module (SAEM) that aggregates contextual semantic features with a deep aggregate group with different receptive fields. Furthermore, to guide the network in learning boundary information for more accurate predictions, we present a Detail Perception Loss (DP Loss) that balances global information and hard samples. Without any additional tricks, extensive experiments demonstrate that our proposed DPSNet achieves real-time performance and significantly surpasses 37 state-of-the-art methods across four standard metrics on four widely used benchmark datasets. The code will be available at https://github.com/fxle/DPSNet.
Recent advancements in deep learning have driven the rapid proliferation of deepfake generation techniques, raising substantial concerns over digital security and trustworthiness. Most current detection methods primarily focus on spatial or frequency domain features but show limited effectiveness when dealing with compressed videos and cross-dataset scenarios. Observing that mainstream generation methods use frame-by-frame synthesis without adequate temporal consistency constraints, we introduce the Spatiotemporal Attention 3D Network (STA-3D), a novel framework that combines a lightweight spatiotemporal attention module with a 3D convolutional architecture to improve detection robustness. The proposed attention module adopts a symmetric multi-branch architecture, where each branch follows a nearly identical processing pipeline to separately model temporal-channel, temporal-spatial, and intra-spatial correlations. Our framework additionally implements Spatial Pyramid Pooling (SPP) layers along the temporal axis, enabling adaptive modeling regardless of input video length. Furthermore, we mitigate the inherent asymmetry in the quantity of authentic and forged samples by replacing standard cross entropy with focal loss for training. This integration facilitates the simultaneous exploitation of inter-frame temporal discontinuities and intra-frame spatial artifacts, achieving competitive performance across various benchmark datasets under different compression conditions: for the intra-dataset setting on FF++, it improves the average accuracy by 1.09 percentage points compared to existing SOTA, with a more significant gain of 1.63 percentage points under the most challenging C40 compression level (particularly for NeuralTextures, achieving an improvement of 4.05 percentage points); while for the intra-dataset setting, AUC is enhanced by 0.24 percentage points on the DFDC-P dataset.