Prohibited item detection in X-ray images is one of the most essential and highly effective methods widely employed in various security inspection scenarios. Considering the significant overlapping phenomenon in X-ray prohibited item images, we propose an anti-overlapping detection transformer (AO-DETR) based on one of the state-of-the-art (SOTA) general object detectors, DETR with improved denoising anchor boxes (DINO). Specifically, to address the feature coupling issue caused by overlapping phenomena, we introduce the category-specific one-to-one assignment (CSA) strategy to constrain category-specific object queries in predicting prohibited items of fixed categories, which can enhance their ability to extract features specific to prohibited items of a particular category from the overlapping foreground–background features. To address the edge blurring problem caused by overlapping phenomena, we propose the look forward densely (LFD) scheme, which improves the localization accuracy of reference boxes in mid-to-high-level decoder layers and enhances the ability to locate blurry edges of the final layer. Similar to DINO, our AO-DETR provides two different versions with distinct backbones, tailored to meet diverse application requirements. Extensive experiments on the PIXray, OPIXray, and HIXray datasets demonstrate that the proposed method surpasses the SOTA object detectors, indicating its potential applications in the field of prohibited item detection. The source code will be available at: https://github.com/Limingyuan001/AO-DETR.

Dual-view baggage inspection has been widely applied in real-world scenarios, where orthogonal viewpoints are deployed to capture diverse and complementary information. Compared with single-view, it can effectively improve the identification performance when rotation and overlay hinder the viewability of the objects. However, this topic has not been rigorously explored due to the scarcity of datasets. To overcome this limitation, we contribute the first fully public large-scale Dual-view X-ray dataset. Our dataset, named DvXray, contains 16,000 pairs, 32,000 X-ray images, in which 15 common classes of 5,496 prohibited items are manually labeled. Besides, we propose an approach named Adaptive Hierarchical Cross Refinement (AHCR) to establish a strong baseline for prohibited item discovery in dual-view X-ray images. AHCR hypothesizes that each input pair is sampled from one mixture distribution, hence gathering the non-overlapping and position-aware cues along the shared axis and complementarily delivering to the other in a hierarchical structure to enrich the feature discriminability of the objects of interest from background overlaps. Upon this structure, we propose an adaptive control strategy and a confidence-weighted view fusion term to make it robust to difficult samples. Extensive experiments on DvXray show that AHCR not only brings significant classification gains over various backbones, such as recent Swin Transformer and ConvNeXt, but also exhibits an impressively better ability to localize objects. In addition, AHCR performs favorably against the counterparts and some recent multi-view learning approaches, moving a step closer towards potential application in practice. Dataset and code are available at https://github.com/Mbwslib/DvXray.

Prohibited item segmentation has a wide range of applications in the security check field, such as computer-aided screening, threat image projection and material discrimination. However, the severe object overlapping in X-ray baggage images restricts the performance of common CNN-based segmentation methods greatly. Worse, no public dataset can be used to promote research in this challenging and promising area. In this paper, to cope with these problems, we present the first Prohibited Item X-ray segmentation dataset named PIXray. PIXray comprises 5,046 X-ray images, in which 15 classes of 14,976 prohibited items are annotated as instance-level masks. Besides, we contribute a dense de-overlap attention snake (DDoAS) in the context of deep learning for automated and real-time prohibited item segmentation. DDoAS mainly includes a dense de-overlap module (DDoM) and an attention deforming module (ADM). Specifically, DDoM is designed to infer prohibited item information accurately from extreme background overlaps through dense reversed connections. ADM aims to improve the low learning efficiency introduced by large variations in shapes and sizes among different prohibited items. Comprehensive evaluation on the PIXray shows the effectiveness and superiority of DDoM and ADM. DDoM excels at recognizing prohibited items from complex backgrounds than other in-domain methods and achieves consistent performance gain over various network backbones, extending the idea of tackling overlapping images data. ADM can ease the model training and further refine the mask quality. Furthermore, out-of-domain experiments prove that DDoAS can also be applied to natural images and achieves comparable performance to the state-of-the-art methods, which implies its potential applications in other fields. The dataset and source code are available at https://github.com/Mbwslib/DDoAS.

Sensor-based Human Activity Recognition (HAR) is widely used in daily life and is the basic-level bridge to virtual healthcare in the metaverse. The current challenge is the low recognition accuracy for personalized users on smart wearable devices. The limited resource cannot support large deep learning models updated locally. Besides, integrating and transmitting sensor data to the cloud would reduce the efficiency. Considering the tradeoff between performance and complexity, we propose a Lightweight Human Activity Recognition (LHAR) framework. In LHAR, we combine the cross-people HAR task with the lightweight model task. LHAR framework is designed on the teacher-student architecture and the student network consists of multiple depthwise separable convolution layers to achieve fewer parameters. The dark knowledge distilled from the complex teacher model enhances the generalization ability of LHAR. To achieve effective knowledge distillation, we propose two optimization methods. Firstly, we train the teacher model by ensemble learning to promote teacher performance. Secondly, a multi-channel data augmentation method is proposed for the diversity of the dataset, which is a plug-in operation for the ensemble teacher model. In the experiments, we compare LHAR with state-of-art models in comparison evaluation, ablation study and the hyperparameter analysis, which proves the better performance of LHAR in efficiency and effectiveness.

A depth perception method combining omnidirectional images and encoding structured light was proposed. First, a new structured light pattern was presented by using monochromatic light. The primitive of the pattern consists of four-direction sand clock-like (FDSC) image. FDSC can provide more robust and accurate position compared with conventional pattern primitive. Second, on the basis of multiple reference planes, a calibration method of projector was proposed to significantly simplify projector calibration in the constructed omnidirectional imaging system. Third, a depth point cloud matching algorithm based on the principle of prior constraint iterative closest point under mobile condition was proposed to avoid the effect of occlusion. The experimental results demonstrated that the proposed method can acquire omnidirectional depth information about large-scale scenes. The error analysis of 16 groups of depth data reported a maximum measuring error of 0.53 mm and an average measuring error of 0.25 mm.

We propose an image‐based face swapping algorithm, which can be used to replace the face in the reference image with the same facial shape and features as the input face. First, a face alignment is made based on a group of detected facial landmarks, so that the aligned input face and the reference face are consistent in size and posture. Secondly, an image warping algorithm based on triangulation is presented to adjust the reference face and its background according to the aligned input faces. In order to achieve more accurate face swapping, a face parsing algorithm is introduced to realize the accurate detection of the face‐ROIs, and then the face‐ROI in the reference image is replaced with the input face‐ROI. Finally, a Poisson image editing algorithm is adopted to realize the boundary processing and color correction between the replacement region and the original background, and then the final face swapping result is obtained. In the experiments, we compare our method with other face swapping algorithms and make a qualitative and quantitative analysis to evaluate the reality and the fidelity of the replaced face. The analysis results show that our method has some advantages in the overall performance of swapping effect.