There are many sensor fusion frameworks proposed in the literature using different sensors and fusion methods combinations and configurations. More focus has been on improving the accuracy performance; however, the implementation feasibility of these frameworks in an autonomous vehicle is less explored. Some fusion architectures can perform very well in lab conditions using powerful computational resources; however, in real-world applications, they cannot be implemented in an embedded edge computer due to their high cost and computational need. We propose a new hybrid multi-sensor fusion pipeline configuration that performs environment perception for autonomous vehicles such as road segmentation, obstacle detection, and tracking. This fusion framework uses a proposed encoder-decoder based Fully Convolutional Neural Network (FCNx) and a traditional Extended Kalman Filter (EKF) nonlinear state estimator method. It also uses a configuration of camera, LiDAR, and radar sensors that are best suited for each fusion method. The goal of this hybrid framework is to provide a cost-effective, lightweight, modular, and robust (in case of a sensor failure) fusion system solution. It uses FCNx algorithm that improve road detection accuracy compared to benchmark models while maintaining real-time efficiency that can be used in an autonomous vehicle embedded computer. Tested on over 3K road scenes, our fusion algorithm shows better performance in various environment scenarios compared to baseline benchmark networks. Moreover, the algorithm is implemented in a vehicle and tested using actual sensor data collected from a vehicle, performing real-time environment perception.

Of all driving functions, one of the critical maneuvers is the lane merge. A cooperative Nonlinear Model Predictive Control (NMPC)-based optimization method for implementing a highway lane merge of two connected autonomous vehicles is presented using solutions obtained by the direct multiple shooting method. A performance criteria cost function, which is a function of the states and inputs of the system, was optimized subject to nonlinear model and maneuver constraints. An optimal formulation was developed and then solved on a receding horizon using direct multiple shooting solutions; this is implemented using an open-source ACADO code. Numerical simulation results were performed in a real-case scenario. The results indicate that the implementation of such a controller is possible in real time, in different highway merge situations.