RADARs and cameras have been present in automotives since the advent of ADAS, as they possess complementary strengths and weaknesses but have been underlooked in the context of learning-based methods. In this work, we propose a method to perform object detection in autonomous driving based on a geometrical and sequential sensor fusion of 3+1D RADAR and semantics extracted from camera data through point cloud painting from the perspective view. To achieve this objective, we adapt PointPainting from the LiDAR and camera domains to the sensors mentioned above. We first apply YOLOv8-seg to obtain instance segmentation masks and project their results to the point cloud. As a refinement stage, we design a set of heuristic rules to minimize the propagation of errors from the segmentation to the detection stage. Our pipeline concludes by applying PointPillars as an object detection network to the painted RADAR point cloud. We validate our approach in the novel View of Delft dataset, which includes 3+1D RADAR data sequences in urban environments. Experimental results show that this fusion is also suitable for RADAR and cameras as we obtain a significant improvement over the RADAR-only baseline, increasing mAP from 41.18 to 52.67 (+27.9%).
Advances in sensor fusion techniques are redefining the landscape of 3D point cloud semantic segmentation, particularly for autonomous driving applications. We propose an enhanced approach that leverages the complementary strengths of LiDAR and multi-camera systems. This study introduces two extensions to the state-of-the-art SalsaNext model [1] based only in LiDAR: SalsaNext+RGB, which integrates RGB data into range-view (RV) images, and SalsaNext+PANO, incorporating panoramic images built from multi-camera setups. The proposed methods are evaluated using the SemanticKITTI and Panoptic nuScenes datasets, showing notable improvements in segmentation accuracy. Results indicate that RGB fusion boosts performance with minimal latency, while panoramic integration offers additional gains at the expense of higher computational load. Comparative analyses highlight significant mIoU gains, demonstrating the potential of multimodal sensor fusion for intricate driving scene understanding.
Traditional routing algorithms optimizing for distance or travel time are inadequate for electric vehicles (EVs), which require energy-aware planning considering battery constraints and charging infrastructure. This work presents an energy-optimal routing system for EVs that integrates personalized consumption modeling with real-time environmental data. The system employs a Long Short-Term Memory (LSTM) neural network to predict State-of-Charge (SoC) consumption from real-world driving data, learning directly from spatiotemporal features including velocity, temperature, road inclination, and traveled distance. Unlike physics-based models requiring difficult-to-obtain parameters, this approach captures nonlinear dependencies and temporal patterns in energy consumption. The routing framework integrates static map data, dynamic traffic conditions, weather information, and charging station locations into a weighted graph representation. Edge costs reflect predicted SoC drops, while node penalties account for traffic congestion and charging opportunities. An enhanced A* algorithm finds optimal routes minimizing energy consumption. Experimental validation on a Nissan Leaf shows that the proposed end-to-end SoC estimator significantly outperforms traditional approaches. The model achieves an RMSE of 36.83 and an of 0.9374, corresponding to a 59.91% reduction in error compared to physics-based formulas. Real-world testing on various routes further confirms its accuracy, with a Mean Absolute Error in the total route SoC estimation of 2%, improving upon the 3.5% observed for commercial solutions.
The implementation of safe and reliable Autonomous Driving Stacks in complex urban environments remains a formidable engineering challenge. While classical modular pipelines provide necessary component-level interpretability, they are inherently rigid, often struggling to adapt to novel environments and failing to provide robust scene interpretation in highly interactive scenarios. In this paper, we present a modular End-to-End ROS-based autonomous driving architecture that upgrades a classical modular baseline by injecting learning-based models into its individual processing layers, integrating GaussianCaR and CLIP for dense semantic BEV perception, expanding the Hierarchical Petri Net state space for safe multi-agent reasoning, refining the planning layer with continuous curve optimization, and replacing the previous reactive controller with an Adaptive Nonlinear Model Predictive Control strategy for superior trajectory tracking. Validated in the CARLA simulator across challenging traffic scenarios and adverse environmental conditions, the proposed architecture raises the Driving Score from 53.81% to 66.46% over the previous baseline, driven by a substantial increase in the Infraction Penalty from 0.59 to 0.79, reflecting a fundamental shift towards safer and more conservative driving behavior at the cost of a moderate reduction in route completion. Against pure End-to-End approaches, our architecture achieves the highest Driving Score at 73.9% and the strongest Infraction Penalty at 0.913, demonstrating that modular interpretability and competitive End-to-End performance are not mutually exclusive. Code will be made publicly available online.