The rapid adoption of Advanced Driver Assistance Systems (ADAS) in modern vehicles, aiming to elevate driving safety and experience, necessitates the real-time processing of high-definition video data. This requirement brings about considerable computational complexity and memory demands, highlighting a critical research void for a design integrating high FPS throughput with optimal Mean Average Precision (mAP) and Mean Intersection over Union (mIoU). Performance improvement at lower costs, multi-tasking ability on a single hardware platform, and flawless incorporation into memory-constrained devices are also essential for boosting ADAS performance. Addressing these challenges, this study proposes an ADAS multi-task learning hardware-software co-design approach underpinned by the Kria KV260 Multi-Processor System-on-Chip Field Programmable Gate Array (MPSoC-FPGA) platform. The approach facilitates efficient real-time execution of deep learning algorithms specific to ADAS applications. Utilizing the BDD100K, KITTI, and CityScapes datasets, our ADAS multi-task learning system endeavours to provide accurate and efficient multi-object detection, segmentation, and lane and drivable area detection in road images. The system deploys a segmentation-based object detection strategy, using a ResNet-18 backbone encoder and a Single Shot Detector architecture, coupled with quantization-aware training to augment inference performance without compromising accuracy. The ADAS multi-task learning offers customization options for various ADAS applications and can be further optimized for increased precision and reduced memory usage. Experimental results showcase the system’s capability to perform real-time multi-class object detection, segmentation, line detection, and drivable area detection on road images at approximately 25.4 FPS using a 1920x1080p Full HD camera. Impressively, the quantized model has demonstrated a 51% mAP for object detection, 56.62% mIoU for image segmentation, 43.86% mIoU for line detection, and 81.56% IoU for drivable area identification, reinforcing its high efficacy and precision. The findings underscore that the proposed ADAS multi-task learning system is a practical, reliable, and effective solution for real-world applications.

This study introduces a new method to enhance ADAS's safety and error prevention capabilities in intelligent vehicles. We address the significant computational and memory demands required for real-time video processing by leveraging BDD100 K, KITTI, CityScape, and Waymo datasets. Our proposed hardware-software co-design integrates an MPSoC-FPGA accelerator for real-time multi-learning models. Our experimental results exhibit that, despite an increase in ADAS tasks and model parameters compared to the state-of-the-art studies, our model achieves 24,715 GOP performance with 4% lower power consumption (6.920 W) and 18.86% less logic resource consumption. The model processes highway scenes at 22.45 FPS and attains 50.06% mAP for object detection, 57.05% mIoU for segmentation, 43.76% mIoU for lane detection, 81.63% IoU for drivable area segmentation, and 9.78% SILog error for depth estimation. These findings confirm the system's effectiveness, reliability, and adaptability for ADAS applications and represent a significant advancement in intelligent vehicle technology, with the potential for further improvements in accuracy and memory efficiency.

This paper introduces a hardware/software co-designed 3D object detection pipeline based on the PointPillars architecture for low-power embedded MPSoC deployment. The proposed system accelerates the computationally intensive stages in programmable logic (PL), including ROI filtering, coordinate transformation, pillarization, centroid extraction, and INT8 neural inference, using Vitis high-level synthesis (HLS) and an integrated Deep Learning Processing Unit (DPU). Control-oriented and irregular operations, such as data acquisition, Direct Memory Access (DMA) control, lightweight Non-Maximum Suppression (NMS), visualization, and logging, remain on the processing system (PS). The design targets the AMD Kria KV260 platform and achieves an accelerated core pipeline latency of 11.4 ms per frame at 300 MHz, corresponding to 87.4 Hz throughput, with 6.842 W board-level power consumption. Including PS-side NMS, the practical end-to-end latency is approximately 12.2 ms for typical KITTI scenes. Compared with existing Field-Programmable Gate Array (FPGA)-based implementations implementations, the proposed design reduces latency by up to $33\times$. It achieves a $202\times$ improvement in on-chip BRAM efficiency across HLS optimization versions through FIFO streaming, dataflow execution, and array partitioning. Experimental validation on physical hardware confirms that the proposed PL-accelerated hardware/software co-design provides a practical and cost-effective solution for real-time 3D LiDAR perception on embedded FPGA platforms.

Achieving a lower memory footprint and reduced computational density in neural network models requires the use of low-precision models. However, existing techniques typically rely on floating-point arithmetic to preserve accuracy, which can be problematic for convolutional neural network models (CNNs) with substantial memory requirements when using floating-point numbers. Additionally, larger bit widths lead to higher computational density in hardware architectures. This has resulted in the need for current models to become deeper network models with sometimes billions, of parameters to address contemporary problems, thereby increasing computational complexity and causing memory allocation issues. These challenges render existing hardware accelerators insufficient. In scenarios where hardware complexity can be traded-off for accuracy, the adoption of model quantization enables the utilization of limited hardware resources for implementing neural networks. From a hardware design standpoint, employing quantized models offers notable advantages in terms of speed, memory utilization, and power consumption compared to traditional floating-point arithmetic. To this end, we propose a method for detecting network intrusions by quantizing weight and activation functions using the Brevitas library in a custom multi-layer detector. We conducted real-time experimentation of the technique on the ZYNQ System-on-Chip (SoC) using the FINN framework, which enabled deep neural network extraction within Field Programmable Gate Arrays (FPGAs), resulting in an accuracy rate of approximately 92%. We selected the UNSW-NB15 dataset, which was generated by the Australian Cyber Security Center (ACCS), for the investigation.