21 April 2025 in IEEE Signal Processing Letters
Detection of interest features is a fundamental pre-processing task for visual odometry and visual simultaneous localization and mapping. The combination of line segments and corners is a trend for those applications, and it is challenging to meet real-time constraints. To pursue this goal, this letter presents a novel points and line points feature detection (PLPD) algorithm designed to be highly efficient and accurate for real-time VO-based systems. The proposed algorithm adopts a novel Multi-level Edge Detector (MED) based on gradient intensity and structure tensor that is not confined solely to extracting corners, but rather, it exploits feature-line points properties to extract long-line segments directly without merging collinear fragment segments. The main focus of this work is to fulfill the real-time and embeddability constraints of our proposed method by adopting the GPU-aware software design for a suitable implementation in GPU-based embedded heterogeneous architecture. Based on this, we exhibit extensive benchmarking with other state-of-the-art algorithms for feature-line segment extraction and corner detection, showing the efficiency of the proposed algorithm and ensuring a real-time performance up to 33 FPS using our dataset.
19 May 2025 in IEEE Embedded Systems Letters
Edge detection is a fundamental aspect of computer vision, facilitating the extraction of crucial features such as corners, edges, and line segments, which are vital for Visual Odometry-based applications. While adding more stages to an edge detector can enhance feature precision, it also increases time consumption and resource requirements, necessitating hardware optimizations. To address these challenges, this paper presents a hardware/software co-design for an efficient pipeline implementation of a Multi-level Edge Detector (MED) on a low-cost FPGA-based heterogeneous architecture. Leveraging this framework, we developed cascading duplicate shift registers based on the Single-Task (ST) mode of OpenCL, which achieves an efficient pipeline execution model for conventional cascade filters. Our ST implementation of the MED significantly outperforms the traditional NDRange implementation in both execution time and resource utilization on the targeted FPGA platform. This allows for effective deployment on the low-cost embedded DE1-SoC, with results validated through comparisons to the NVIDIA Jetson Nano GPU.
14 September 2023 in TechRxiv
This paper presents a novel feature-line points tracking algorithm designed to be highly efficient and accurate for real-time monocular line-based SfM applications in man-made environments. The proposed algorithm exploits feature-line points proprieties to detect and extract long line segments and ensure their temporal evolution in the subsequent frames while handling line segment detection and tracking issues such as noisy images, over-segmentation, edge thresholding, false detections, occlusions, and ambiguous matches. To fulfill the real-time and embeddability constraints of our proposed method, we adopt the hardware/software co-design to achieve a suitable implementation with a scalable FPGA-based embedded heterogeneous architecture. Based on this, experimental results show the portability of the line segment detector block of this proposed algorithm on scalable FPGA-based embedded heterogeneous architectures, improving the acceleration of the sequential execution by about 98\%, whereas a 53\% improvement with GPU-based heterogeneous architectures. For qualitative and quantitative assessment, we exhibit extensive benchmarking with other state-of-the-art algorithms regarding feature-line segment extraction and tracking, showing the efficiency of the proposed algorithm and ensuring very hopeful real-time performance using two common datasets.
14 April 2026 in SSRN Electronic Journal
Visual Inertial Odometry (VIO) has become a key technology for motion estimation in robotics and autonomous systems. However, deploying accurate VIO pipelines on embedded platforms remains challenging due to the trade-off between estimation accuracy, real-time performance, and energy consumption. This paper presents a hardware-aware Feature-PLPD VIO framework that integrates a point-and-line visual front-end with a loosely coupled Error-State Extended Kalman Filter (ESEKF) back-end. The proposed approach follows an Algorithm Architecture Adequacy (A3) approach to ensure computational efficiency while preserving robustness. To demonstrate scalability, two end-to-end embedded implementations targeting heterogeneous architectures are developed. The first is a stereo Feature-PLPD VIO system deployed on a GPU-based platform using GPU-aware software design, while the second is a low-cost monocular implementation on an FPGA-based architecture using hardware–software co-design with depth scale correction. The systems are validated through offline experiments on benchmark datasets, and the stereo implementation is further evaluated in real-time on-the-fly on a robotic platform. Experimental results show that the proposed framework achieves accurate and real-time motion estimation under strict resource constraints. In particular, the system runs at approximately 20 FPS and achieves meter-level trajectory accuracy on outdoor using KITTI dataset, with about 17\% improvement over VO-only methods. Real-time on-the-fly deployment on a Scout Mini robot in an indoor VICON-equipped environment further demonstrates the feasibility of scalable embedded VIO systems for resource-constrained robotic applications.