LiDAR has become a vital sensor for autonomous driving scene understanding. To meet the accuracy and speed of LiDAR point clouds semantic segmentation, an efficient model ACPNet is proposed in this paper. In the feature extraction stage, the backbone is constructed with asymmetric convolutions, so the skeleton of the square convolution kernel is enhanced, which leads to greater robustness to target rotation. Moreover, a contextual feature enhancement module is designed to extract richer contextual features. During training, global scaling and global translation are performed to enrich the diversity of datasets. Compared with the baseline network PolarNet, the mIoU of ACPNet on the SemanticKITTI, SemanticPOSS and nuScenes datasets are improved by 5.1%, 1.6% and 2.9%, respectively. Meanwhile, the speed of ACPNet is 14 FPS, which basically meets the real-time requirements in autonomous driving scenarios. The experimental results show that ACPNet significantly improves the performance of LiDAR point cloud semantic segmentation.
10 January 2023 in Signal, Image and Video Processing
With the application of the random sampling method in the down-sampling of point clouds data, the processing speed of point clouds has been greatly improved. However, the utilization of semantic information is still insufficient. To address this problem, we propose a point cloud semantic segmentation network called MFFRand (Multi-Scale Feature Fusion Based on RandLA-Net). Based on RandLA-Net, a multi-scale feature fusion module is developed, which is stacked by encoder-decoders with different depths. The feature maps extracted by the multi-scale feature fusion module are continuously concatenated and fused. Furthermore, for the network to be trained better, the multi-loss supervision module is proposed, which could strengthen the control of the training process of the local structure by adding sub-losses in the end of different decoder structures. Moreover, the trained MFFRand network could be connected to the inference network by different decoder terminals separately, which could achieve the inference of different depths of the network. Compared to RandLA-Net, MFFRand has improved mIoU on both S3DIS and Semantic3D datasets, reaching 71.1% and 74.8%, respectively. Extensive experimental results on the point cloud dataset demonstrate the effectiveness of our method.
High-accuracy detection of vacant parking slots using visual system is a prerequisite for automatic parking, which is significant for the promotion of automatic parking. Complex visual environment such as light occlusion, intensity variations and visual accuracy raises significant challenges for vision-based system parking slot detection. To address such problems, REPSDet (Resnet50 based Parking Slot Detection) is proposed for parking slot detection and occupancy classification in this paper, which is based on deep learning. Parking slot proximity information could be fully extracted by REPSDet,which is combined with parking slot classification and location information to improve the parking slot detection effect. For parking slot detection, REPS-Net is proposed in this paper, which is an improved yolov4 parking slot detection method. In REPS-Net, marker point positioning combined with geometric clues are used to infer the precise location of parking slots. For occupancy classification, the deep convolutional network m\_DCNN is proposed in this paper. Network structure and operation performance are optimized by m_DCNN, and the accuracy of parking slot detection is further improved. Experimental results show that REPSDet achieves precision rate of 97.77% and recall rate of 99.38% on the PS2.0_A dataset, which is significantly better than compared methods.
1 January 2024 in Journal of Applied Remote Sensing
Fusing LiDAR point cloud and camera image for 3D object detection in autonomous driving has emerged as a captivating research avenue. The core challenge of multimodal fusion is how to seamlessly fuse 3D LiDAR point cloud with 2D camera image. Although current approaches exhibit promising results, they often rely solely on fusion at either the data level, feature level, or object level, and there is still a room for improvement in the utilization of multimodal information. We present an advanced and effective multimodal fusion framework called EPAWFusion for fusing 3D point cloud and 2D camera image at both data level and feature level. EPAWFusion model consists of three key modules: a point enhanced module based on semantic segmentation for data-level fusion, an adaptive weight allocation module for feature-level fusion, and a detector based on 3D sparse convolution. The semantic information of the 2D image is extracted using semantic segmentation, and the calibration matrix is used to establish the point-pixel correspondence. The semantic information and distance information are then attached to the point cloud to achieve data-level fusion. The geometry features of enhanced point cloud are extracted by voxel encoding, and the texture features of image are obtained using a pretrained 2D CNN. Feature-level fusion is achieved via the adaptive weight allocation module. The fused features are fed into a 3D sparse convolution-based detector to obtain the accurate 3D objects. Experiment results demonstrate that EPAWFusion outperforms the baseline network MVXNet on the KITTI dataset for 3D detection of cars, pedestrians, and cyclists by 5.81%, 6.97%, and 3.88%. Additionally, EPAWFusion performs well for single-vehicle-side 3D object detection based on the experimental findings on DAIR-V2X dataset and the inference frame rate of our proposed model reaches 11.1 FPS. The two-layer level fusion of EPAWFusion significantly enhances the performance of multimodal 3D object detection.
This paper presents MicroRCA-Agent, an innovative solution for microservice root cause analysis based on large language model agents, which constructs an intelligent fault root cause localization system with multimodal data fusion. The technical innovations are embodied in three key aspects: First, we combine the pre-trained Drain log parsing algorithm with multi-level data filtering mechanism to efficiently compress massive logs into high-quality fault features. Second, we employ a dual anomaly detection approach that integrates Isolation Forest unsupervised learning algorithms with status code validation to achieve comprehensive trace anomaly identification. Third, we design a statistical symmetry ratio filtering mechanism coupled with a two-stage LLM analysis strategy to enable full-stack phenomenon summarization across node-service-pod hierarchies. The multimodal root cause analysis module leverages carefully designed cross-modal prompts to deeply integrate multimodal anomaly information, fully exploiting the cross-modal understanding and logical reasoning capabilities of large language models to generate structured analysis results encompassing fault components, root cause descriptions, and reasoning trace. Comprehensive ablation studies validate the complementary value of each modal data and the effectiveness of the system architecture. The proposed solution demonstrates superior performance in complex microservice fault scenarios, achieving a final score of 50.71. The code has been released at: https://github.com/tangpan360/MicroRCA-Agent.