Prohibited item detection in X-ray images is one of the most essential and highly effective methods widely employed in various security inspection scenarios. Considering the significant overlapping phenomenon in X-ray prohibited item images, we propose an anti-overlapping detection transformer (AO-DETR) based on one of the state-of-the-art (SOTA) general object detectors, DETR with improved denoising anchor boxes (DINO). Specifically, to address the feature coupling issue caused by overlapping phenomena, we introduce the category-specific one-to-one assignment (CSA) strategy to constrain category-specific object queries in predicting prohibited items of fixed categories, which can enhance their ability to extract features specific to prohibited items of a particular category from the overlapping foreground–background features. To address the edge blurring problem caused by overlapping phenomena, we propose the look forward densely (LFD) scheme, which improves the localization accuracy of reference boxes in mid-to-high-level decoder layers and enhances the ability to locate blurry edges of the final layer. Similar to DINO, our AO-DETR provides two different versions with distinct backbones, tailored to meet diverse application requirements. Extensive experiments on the PIXray, OPIXray, and HIXray datasets demonstrate that the proposed method surpasses the SOTA object detectors, indicating its potential applications in the field of prohibited item detection. The source code will be available at: https://github.com/Limingyuan001/AO-DETR.

Skeleton-based human action recognition has received extensive attention due to its efficiency and robustness to complex backgrounds. Though the human skeleton can accurately capture the dynamics of human poses, it fails to recognize human actions induced by the interaction between human and objects, making it is of great importance to further explore the interaction between the human and objects for human action recognition. In this paper, we devise the multi-stream interaction networks (MSIN), to simultaneously explore the dynamics of human skeleton, objects, and the interaction between human and objects. Specifically, apart from the traditional human skeleton stream, 1) the second stream explores the dynamics of object appearance from the objects surrounding the human body joints; and 2) the third stream captures the dynamics of object position in regard to the distance between the object and different human body joints. Experimental results on three popular skeleton-based human action recognition datasets, NTU RGB + D, NTU RGB + D 120, and SYSU, demonstrate the effectiveness of the proposed method, especially for recognizing the human actions with human-object interactions.

Prohibited item segmentation has a wide range of applications in the security check field, such as computer-aided screening, threat image projection and material discrimination. However, the severe object overlapping in X-ray baggage images restricts the performance of common CNN-based segmentation methods greatly. Worse, no public dataset can be used to promote research in this challenging and promising area. In this paper, to cope with these problems, we present the first Prohibited Item X-ray segmentation dataset named PIXray. PIXray comprises 5,046 X-ray images, in which 15 classes of 14,976 prohibited items are annotated as instance-level masks. Besides, we contribute a dense de-overlap attention snake (DDoAS) in the context of deep learning for automated and real-time prohibited item segmentation. DDoAS mainly includes a dense de-overlap module (DDoM) and an attention deforming module (ADM). Specifically, DDoM is designed to infer prohibited item information accurately from extreme background overlaps through dense reversed connections. ADM aims to improve the low learning efficiency introduced by large variations in shapes and sizes among different prohibited items. Comprehensive evaluation on the PIXray shows the effectiveness and superiority of DDoM and ADM. DDoM excels at recognizing prohibited items from complex backgrounds than other in-domain methods and achieves consistent performance gain over various network backbones, extending the idea of tackling overlapping images data. ADM can ease the model training and further refine the mask quality. Furthermore, out-of-domain experiments prove that DDoAS can also be applied to natural images and achieves comparable performance to the state-of-the-art methods, which implies its potential applications in other fields. The dataset and source code are available at https://github.com/Mbwslib/DDoAS.

Dual-view baggage inspection has been widely applied in real-world scenarios, where orthogonal viewpoints are deployed to capture diverse and complementary information. Compared with single-view, it can effectively improve the identification performance when rotation and overlay hinder the viewability of the objects. However, this topic has not been rigorously explored due to the scarcity of datasets. To overcome this limitation, we contribute the first fully public large-scale Dual-view X-ray dataset. Our dataset, named DvXray, contains 16,000 pairs, 32,000 X-ray images, in which 15 common classes of 5,496 prohibited items are manually labeled. Besides, we propose an approach named Adaptive Hierarchical Cross Refinement (AHCR) to establish a strong baseline for prohibited item discovery in dual-view X-ray images. AHCR hypothesizes that each input pair is sampled from one mixture distribution, hence gathering the non-overlapping and position-aware cues along the shared axis and complementarily delivering to the other in a hierarchical structure to enrich the feature discriminability of the objects of interest from background overlaps. Upon this structure, we propose an adaptive control strategy and a confidence-weighted view fusion term to make it robust to difficult samples. Extensive experiments on DvXray show that AHCR not only brings significant classification gains over various backbones, such as recent Swin Transformer and ConvNeXt, but also exhibits an impressively better ability to localize objects. In addition, AHCR performs favorably against the counterparts and some recent multi-view learning approaches, moving a step closer towards potential application in practice. Dataset and code are available at https://github.com/Mbwslib/DvXray.

Anomaly detection in crowded scenes is an important and challenging part of the intelligent video surveillance system. As the deep neural networks make success in feature representation, the features extracted by a deep neural network represent the appearance and motion patterns in different scenes more specifically, comparing with the hand-crafted features typically used in the traditional anomaly detection approaches. In this paper, we propose a new baseline framework of anomaly detection for complex surveillance scenes based on a variational auto-encoder with convolution kernels to learn feature representations. Firstly, the raw frames series are provided as input to our variational auto-encoder without any preprocessing to learn the appearance and motion features of the receptive fields. Then, multiple Gaussian models are used to predict the anomaly scores of the corresponding receptive fields. Our proposed two-stage anomaly detection system is evaluated on the video surveillance dataset for a large scene, UCSD pedestrian datasets, and yields competitive performance compared with state-of-the-art methods.

Sensor-based Human Activity Recognition (HAR) is widely used in daily life and is the basic-level bridge to virtual healthcare in the metaverse. The current challenge is the low recognition accuracy for personalized users on smart wearable devices. The limited resource cannot support large deep learning models updated locally. Besides, integrating and transmitting sensor data to the cloud would reduce the efficiency. Considering the tradeoff between performance and complexity, we propose a Lightweight Human Activity Recognition (LHAR) framework. In LHAR, we combine the cross-people HAR task with the lightweight model task. LHAR framework is designed on the teacher-student architecture and the student network consists of multiple depthwise separable convolution layers to achieve fewer parameters. The dark knowledge distilled from the complex teacher model enhances the generalization ability of LHAR. To achieve effective knowledge distillation, we propose two optimization methods. Firstly, we train the teacher model by ensemble learning to promote teacher performance. Secondly, a multi-channel data augmentation method is proposed for the diversity of the dataset, which is a plug-in operation for the ensemble teacher model. In the experiments, we compare LHAR with state-of-art models in comparison evaluation, ablation study and the hyperparameter analysis, which proves the better performance of LHAR in efficiency and effectiveness.