18 March 2024 in IEEE Transactions on Vehicular Technology
The emergence of edge computing in Intelligent Transportation Systems (ITS) has shown promise in enabling real-time sensor fusion applications. In this paper, we explore how to utilize edge computing to aid in collaborative vehicular perception, an emerging topic within ITS which involves vehicles sharing sensor data with one another to extend each vehicle's perception beyond what its individual sensors can see. However, achieving real time collaborative perception is a challenge even with the utilization of edge computing; due to the large amount of multi-modal sensor data produced by modern intelligent vehicles, the amount of data transmitted over wireless channels and the complexity of computational tasks will need to be managed dynamically based on the wireless network conditions and available computing resources. As such we propose REFO, a Real-time Edge Fusion Optimization method that combines task partitioning with data reduction and model compression to maximize sensor fusion accuracy while adhering to Quality of Service (QoS) requirements. We define a performance metric termed effective mean average precision (EmAP), which incorporates both QoS and fusion accuracy, and show that our proposed neural network based REFO action decision framework can outperform the best comparison models by approximately 10% in terms of EmAP over 18 test combinations of network and computing conditions.
24 January 2025 in IEEE Open Journal of Vehicular Technology
Vehicular sensing has reached new heights due to advances in external perception systems enabled by the increasing number and type of sensors in vehicles, as well as the availability of on-board computing. These changes have led to improvements in driver safety and have also created a highly heterogeneous environment of vehicles on the road today in terms of sensing and computing. Using collaborative perception, the information obtained by vehicles with sensing capabilities can be expanded and improved, and older vehicles that lack external sensors and computing capabilities can be informed of potential hazards, opening the opportunity to improve traffic efficiency and safety on the roads. However, achieving real-time collaborative perception is a difficult task due to the dynamic availability of vehicular sensing and computing and the highly variable nature of vehicular communications. To address these challenges, we propose a Heterogeneous Adaptive Collaborative Perception (HAdCoP) framework which utilizes a Context-aware Latency Prediction Network (CaLPeN) to intelligently select which vehicles should transmit their sensor data, the specific individual and collaborative perception tasks, and the amount of computational offloading that should be utilized given information about the current state of the environment. Additionally, we propose an Adaptive Perception Frequency (APF) model to determine the optimal end-to-end latency requirement according to the current state of the environment. The proposed CaLPeN model outperforms six implemented comparison models in terms of effective mean average precision (EmAP), beating the next best model's performance by 5.5% on average when tested on the OPV2V perception dataset using two different combinations of wireless communication conditions and vehicular sensor/computing distributions.
12 December 2022 in IEEE Access
Improvements in vehicular perception systems over the last decade have enabled new levels of safety and awareness in modern production vehicles. However, achievable performance of these perception systems is bounded by sensor limitations, such as range, and environmental factors, such as occlusion. Collaborative perception circumvents these limitations by incorporating sensor data from multiple sources to fill in perception gaps experienced by an individual sources’ sensors. This paper explores one important aspect of collaborative perception: simultaneously associating objects detected by multiple individual vehicles with each other. This task is crucial as the inability to perform such object association accurately can lead to duplicate or missed detections, which can lead to unsafe driving behavior. This work proposes a graph neural network model for this task that achieves an average precision (AP) of 0.882 in a challenging virtual environment consisting of 25 unique, simultaneous, and mobile viewpoints. A simpler real-world scenario with two static viewpoints is also evaluated where the model achieves an AP of 0.998, showing that this model can readily transfer to real-world scenarios as well.