Current state-of-the-art (SOTA) LiDAR-only detectors perform well for 3D object detection tasks, but point cloud data are typically sparse and lacks semantic information. Detailed semantic information obtained from camera images can be added with existing LiDAR-based detectors to create a robust 3D detection pipeline. With two different data types, a major challenge in developing multi-modal sensor fusion networks is to achieve effective data fusion while managing computational resources. With separate 2D and 3D feature extraction backbones, feature fusion can become more challenging as these modes generate different gradients, leading to gradient conflicts and suboptimal convergence during network optimization. To this end, we propose a 3D object detection method, Attention-Enabled Point Fusion (AEPF). AEPF uses images and voxelized point cloud data as inputs and estimates the 3D bounding boxes of object locations as outputs. An attention mechanism is introduced to an existing feature fusion strategy to improve 3D detection accuracy and two variants are proposed. These two variants, AEPF-Small and AEPF-Large, address different needs. AEPF-Small, with a lightweight attention module and fewer parameters, offers fast inference. AEPF-Large, with a more complex attention module and increased parameters, provides higher accuracy than baseline models. Experimental results on the KITTI validation set show that AEPF-Small maintains SOTA 3D detection accuracy while inferencing at higher speeds. AEPF-Large achieves mean average precision scores of 91.13, 79.06, and 76.15 for the car class’s easy, medium, and hard targets, respectively, in the KITTI validation set. Results from ablation experiments are also presented to support the choice of model architecture.
Safe autonomous vehicle (AV) operations depend on an accurate perception of the driving environment, which necessitates the use of a variety of sensors. Computational algorithms must then process all of this sensor data, which typically results in a high on-vehicle computational load. For example, existing lane markings are designed for human drivers, can fade over time, and can be contradictory in construction zones, which require specialized sensing and computational processing in an AV. But, this standard process can be avoided if the lane information is simply transmitted directly to the AV. High definition maps and road side units (RSUs) can be used for direct data transmission to the AV, but can be prohibitively expensive to establish and maintain. Additionally, to ensure robust and safe AV operations, more redundancy is beneficial. A cost-effective and passive solution is essential to address this need effectively. In this research, we propose a new infrastructure information source (IIS), chip-enabled raised pavement markers (CERPMs), which provide environmental data to the AV while also decreasing the AV compute load and the associated increase in vehicle energy use. CERPMs are installed in place of traditional ubiquitous raised pavement markers along road lane lines to transmit geospatial information along with the speed limit using long range wide area network (LoRaWAN) protocol directly to nearby vehicles. This information is then compared to the Mobileye commercial off-the-shelf traditional system that uses computer vision processing of lane markings. Our perception subsystem processes the raw data from both CEPRMs and Mobileye to generate a viable path required for a lane centering (LC) application. To evaluate the detection performance of both systems, we consider three test routes with varying conditions. Our results show that the Mobileye system failed to detect lane markings when the road curvature exceeded m−1. For the steep curvature test scenario, it could only detect lane markings on both sides of the road for just 6.7% of the given test route. On the other hand, the CERPMs transmit the programmed geospatial information to the perception subsystem on the vehicle to generate a reference trajectory required for vehicle control. The CERPMs successfully generated the reference trajectory for vehicle control in all test scenarios. Moreover, the CERPMs can be detected up to 340 m from the vehicle’s position. Our overall conclusion is that CERPM technology is viable and that it has the potential to address the operational robustness and energy efficiency concerns plaguing the current generation of AVs.