In the autonomous driving industry, there is a growing trend to employ long-wave infrared (LWIR)-based uncooled thermal-imaging cameras, capable of robustly collecting data even in extreme environments. Consequently, both industry and academia are actively researching contrast-enhancement techniques to improve the quality of LWIR-based thermal-imaging cameras. However, most research results only showcase experimental outcomes using mass-produced products that already incorporate contrast-enhancement techniques. Put differently, there is a lack of experimental data on contrast enhancement post-non-uniformity (NUC) and temperature compensation (TC) processes, which generate the images seen in the final products. To bridge this gap, we propose a histogram equalization (HE)-based contrast enhancement method that incorporates a region-based clipping technique. Furthermore, we present experimental results on the images obtained after applying NUC and TC processes. We simultaneously conducted visual and qualitative performance evaluations on images acquired after NUC and TC processes. In the visual evaluation, it was confirmed that the proposed method improves image clarity and contrast ratio compared to conventional HE-based methods, even in challenging driving scenarios such as tunnels. In the qualitative evaluation, the proposed method demonstrated upper-middle-class rankings in both image quality and processing speed metrics. Therefore, our proposed method proves to be effective for the essential contrast enhancement process in LWIR-based uncooled thermal-imaging cameras intended for autonomous driving platforms.
Face recognition applications are being widely studied owing to their extensive usability in the field of computer vision. However, processing an entire image requires a considerable amount of time. To reduce the processing time, several algorithms that extract only the face from the image during pre-processing are studied. Haar classifiers are extensively used for the hardware implementation of face detection algorithms that improve the processing speed of face classification. This paper proposes a Haar classifier based face detection architecture that removes unnecessary iterations during classification to further improve the processing speed. The proposed architecture improves the processing speed by 4.46% compared to that of conventional Haar classifier based face detection architectures, for face detection using a VGA image with 30 faces. The proposed architecture tends to improve the processing speed as the number of faces in the image increases while matching the detection accuracy of conventional methods. Additionally, this architecture can be widely applied to classification algorithms that are based on iterations.
This paper presents an integrated image processor architecture designed for realtime interfacing and processing of high-resolution thermal video obtained from an uncooled infrared focal plane array (IRFPA) utilizing a modern system-on-chip field-programmable gate array (SoC FPGA). Our processor provides a one-chip solution for incorporating non-uniformity correction (NUC) algorithms and contrast enhancement methods (CEM) to be performed seamlessly. We have employed NUC algorithms that utilize multiple coefficients to ensure robust image quality, free from ghosting effects and blurring. These algorithms include polynomial modeling-based thermal drift compensation (TDC), two-point correction (TPC), and runtime discrete flat field correction (FFC). To address the memory bottlenecks originating from the parallel execution of NUC algorithms in realtime, we designed accelerators and parallel caching modules for pixel-wise algorithms based on a multi-parameter polynomial expression. Furthermore, we designed a specialized accelerator architecture to minimize the interrupted time for runtime FFC. The implementation on the XC7Z020CLG400 SoC FPGA with the QuantumRed VR thermal module demonstrates that our image processing module achieves a throughput of 60 frames per second (FPS) when processing 14-bit 640×480 resolution infrared video acquired from an uncooled IRFPA.
In stereo vision systems, mismatching can occur frequently in specific regions containing noise and high-frequency components, such as checkered patterns. Therefore, this paper proposes a preprocessing method and architecture based on a 2-D Haar filter to improve the matching accuracy of disparity map by reducing the high-frequency and noise components. In this paper, the disparity map is computed by the semi-global matching (SGM) method, and separable weighted median filter is adopted as postprocessing. The SGM with proposed method reduces the average number of mismatching pixels by 3.5233% in non-occlusion condition and 3.2647% in occlusion condition compared to that with a 2-D Gaussian filter when using the KITTI 2015 stereo dataset. The proposed method is suitable for embedded stereo vision systems that requires high matching accuracy with reasonable resource overhead.
Embedded stereo vision systems based on traditional approaches often require a disparity refinement process to enhance image quality. Weighted median filter (WMF)-based processors are commonly employed for their excellent refinement performance. However, when implemented on a field-programmable gate array (FPGA), WMF-based processors face a trade-off between hardware resource utilization and refinement performance. To address this trade-off, we previously proposed a new disparity refinement processor based on the hybrid max-median filter (HMMF). However, our earlier work did not guarantee flawless operation in large occluded and texture-less regions, particularly in areas with numerous holes. In order to overcome this limitation of conventional processors, we proposed a cell-based disparity refinement processor. This processor extends our previous HMMF-based disparity refinement processor. To evaluate its refinement performance, we conducted experiments using four types of publicly available stereo datasets. When comparing refinement performance, our proposed processor outperforms conventional processors when using the KITTI 2012 and 2015 stereo benchmark datasets. Additionally, the results demonstrate that our proposed processor exhibits superior refinement performance when applied to the Cityscapes and StereoDriving datasets in comparison to conventional processors. Furthermore, when considering hardware resource utilization, our proposed processor demonstrates lower resource requirements than conventional processors when implemented on an FPGA. Therefore, our proposed disparity refinement processor is well-suited for the disparity refinement process in stereo vision systems that require cost-effectiveness and high performance.