2 June 2025 in IEEE Transactions on Aerospace and Electronic Systems
Human action recognition (HAR) methods based on ultra-wideband (UWB) multiple-input multiple-output (MIMO) radar have demonstrated substantial potential in complex environments. However, the use of single-modal data, such as spectrograms or point clouds, poses significant limitations; it is easy to get confused when dealing with similar actions, particularly in challenging scenarios. To overcome these limitations, this study proposes an action recognition approach that leverages bimodal data and spatiotemporal features for improved performance. Four-dimensional radar image data of the human body are captured using UWB MIMO radar and processed into four-dimensional point cloud data and four-dimensional intensity projection image data, which collectively serve as bimodal feature inputs. To process these bimodal feature data, a network architecture named Point Image Fusion-Net (PIF-Net) is developed, incorporating spatiotemporal sequence classification to achieve recognition of human actions. The network initially extracts spatial features from the bimodal feature data, followed by temporal feature extraction, culminating in the classification of spatiotemporal features. Experimental results show that the proposed method achieves an accuracy of 96.75% in action recognition tasks, validating the approach.
18 March 2025 in IEEE Transactions on Instrumentation and Measurement
This paper proposes a multimodal human pose re-construction method based on 3D ultra-wideband (UWB) radar images and point clouds, aiming to improve the accuracy of human pose estimation through the fusion of radar images and point cloud data. First, a UWB 3D imaging radar system is designed, which synchronously collects radar echo signals and optical images, con-structing a multimodal dataset covering various common actions and different human characteristics. Radar data processing in-cludes azimuth-range 2D imaging, target locking, local 3D imag-ing, discrete sampling, and maximum projection to generate point cloud data and projection images. Optical image processing uses mature methods to reconstruct 3D poses as pose labels for point clouds and projection images. To achieve multimodal data fusion, the UWB FusionPose network is designed, comprising an image feature extraction network, a point cloud feature extraction net-work, and a pose reconstruction network. The image feature ex-traction network is based on the ResNet-18 framework, while the point cloud feature extraction network adopts a pyramid struc-ture. After feature fusion, a multi-layer perceptron (MLP) is used to predict human pose information. Additionally, this paper ex-plores the impact of fusion parameters on network performance and verifies the effectiveness of the multimodal network through ablation experiments. Experimental results show that this method effectively utilizes radar point cloud data and projection image data to accurately reconstruct the 3D pose of human targets. This research not only provides a new human pose reconstruction tech-nique but also offers valuable references for the future develop-ment of radar imaging technology and multimodal data fusion methods.