1 January 2021 in Computational Intelligence and Neuroscience
Gesture recognition is one of the important ways of human‐computer interaction, which is mainly detected by visual technology. The temporal and spatial features are extracted by convolution of the video containing gesture. However, compared with the convolution calculation of a single image, multiframe image of dynamic gestures has more computation, more complex feature extraction, and more network parameters, which affects the recognition efficiency and real‐time performance of the model. To solve above problems, a dynamic gesture recognition model based on CBAM‐C3D is proposed. Key frame extraction technology, multimodal joint training, and network optimization with BN layer are used for making the network performance better. The experiments show that the recognition accuracy of the proposed 3D convolutional neural network combined with attention mechanism reaches 72.4% on EgoGesture dataset, which is improved greatly compared with the current main dynamic gesture recognition methods, and the effectiveness of the proposed algorithm is verified.
26 June 2023 in IEEE Transactions on Intelligent Transportation Systems
Autonomous driving has gradually become a research hotspot in recent years. Visual Simultaneous Localization and Mapping (SLAM) technology can help unmanned vehicles accurately explore the environment at a lower cost, and the readability of the map can be improved by integrating target detection algorithms. However, the location and 3D shape of the object in the map were not obtained. The method of RGBD-SLAM based on object detection with two-stream YOLOv4-MobileNetv3 convolutional neural network is proposed in this paper. RGBD SLAM algorithm and target detection algorithm are combined to build an algorithm model that can generate the global sparse map and build target dense map quickly. The two-stream network is integrated to obtain 2D information about the target, and further combined with the camera pose after the front-end key frame detection of the SLAM algorithm in this paper, and the dense 3D point cloud of the target and the center point position of the object is obtained. Then, the sparse point cloud of the SLAM system and the dense point cloud of the target can be obtained. The experimental results show that the number of point clouds decreases by about 50% and the time for mapping accounts for about 60% of the global dense mapping time. The method of this paper can efficiently decrease the computational space and improve the speed of semantic mapping, which verifies its feasibility and superiority. It can be used to achieve large-area mapping and the ability to update maps during autonomous driving.
30 September 2021 in Concurrency and Computation: Practice and Experience
16 November 2022 in IEEE Sensors Journal
With the rapid development of artificial intelligence, neural network is widely used in various fields. Target detection algorithm is mainly based on neural network, but the accuracy of target detection algorithm is greatly related to the complexity of scene and texture. A target detection algorithm based on RGB-D image from the perspective of the lightweight of target detection network model and the integration of depth map to overcome the weak environmental illumination with self-powered sensors information is proposed. This paper analyzes the network model structure of YOLOv4 and MobileNet, compares the variation of parameter number between depthwise separable convolution and convolutional neural network, and combines the advantages of YOLOv4 network and MobileNetv3 network. The main network of three effective feature layers in YOLOv4 is replaced by MobileNetv3 network for initial feature layer extraction to strengthen the feature extraction network. At the same time, the standard convolution models in the network are replaced by depthwise separable convolution. The proposed method is compared with YOLOv4 and YOLOv4-MobileNetv3 in this paper, and the experimental results show that the proposed network retains its original advantages in accuracy, but the size of the network model is about 23% of that of YOLOv4 network model, and the processing speed is about 42% higher than that of YOLOv4 network model, and the detection accuracy can still reach 83% in the environment with poor lighting conditions.