In the natural and practical scenario, the captured video sequence under bad weather situations or low light conditions often suffers from poor visibility and low-contrast problems. This hurts the performance of the high-level processing, e.g. object tracking or recognition. In this paper, we develop an FPGA-based low-visibility enhancement accelerator for video sequence by adaptive histogram equalization with dynamic clip-threshold (AHEwDC) which is determined by the visibility assessment. The main goal is to improve the low visibility with high image quality for both hazy and low-light video sequences in real-time. Firstly, a concept to quantify the visual perception based on supervised learning is to estimate the visibility score. Then, to avoid the problem of noise amplification in the conventional method, we propose a visibility assessment model to find an optimal clip-threshold. The contrast energy of gray channel, yellow-blue channel and red-green channel, average saturation, and gradients are statistical features in the model to describe the visibility of an image. Finally, to meet the speed requirement for video sequence processing, a specified hardware architecture for both visibility assessment and AHEwDC is implemented on FPGA. Besides, a mean spatial filter for cumulative distribution functions (CDFs) of the AHE is developed for suppressing the noise caused by a single-color local region. The demonstration system on the DE1-SoC platform with the Intel Cyclone V FPGA device with the max working frequency of 75.84 MHz is capable of processing 30 fps FHD (1920x 1080) video.

The Hough Transform (HT) is a method for extracting straight lines from an edge image. The main limitations of the HT for usage in actual applications are computation time and storage requirements. This paper reports a hardware architecture for HT implementation on a Field Programmable Gate Array (FPGA) with parallelized voting procedure. The 2-dimensional accumulator array, namely the Hough space in parametric form (ρ, θ), for computing the strength of each line by a voting mechanism is mapped on a 1-dimensional array with regular increments of θ. Then, this Hough space is divided into a number of parallel parts. The computation of (ρ, θ) for the edge pixels and the voting procedure for straight-line determination are therefore executable in parallel. In addition, a synchronized initialization for the Hough space further increases the speed of straight-line detection, so that XGA video processing becomes possible. The designed prototype system has been synthesized on a DE4 platform with a Stratix-IV FPGA device. In the application of road-lane detection, the average processing speed of this HT implementation is 5.4ms per XGA-frame at 200 MHz working frequency.

Significantly improved performance of the various learning algorithms has revived the interest in computer vision for recognition applications during the current decade. This paper reports a vision-based hardware recognition architecture combining the Haar-like feature extraction with the support vector machine (SVM) classification. To support an optimal trade-off between resource requirements, processing speed, and recognition accuracy, a 12-bit fixed-point computation for block-based feature normalization and a recycling allocation of minimalized memory resources are proposed in this work. Furthermore, an efficient scale generation of target objects for recognition is enabled by configurable windows with high size flexibility. Additionally, a parallel-partial SVM-classification architecture is developed for improving the recognition speed, by accumulating partially completed SVM results for multiple windows in parallel. The proposed hardware architecture is verified with an Altera DE4 platform to achieve a high throughput rate of 216 fps and 70 fps for XGA (1024×768) and HD (1920×1080) video resolutions, respectively. A recycled memory space of only 193 kb is sufficient for processing high-resolution images up to 2048×2048 pixels during on-line testing. Using the INRIA person dataset, 89.81% average precision (AP) and a maximum accuracy of 96.93% for pedestrian recognition are realized. Further, about 99.08% accuracy is achieved for two car recognition tasks using the UIUC dataset (side view of cars) and a frontal car dataset collected by ourselves at Hiroshima University (HU) with the proposed hardware-architecture framework.

The semi-global matching (SGM) algorithm in stereo vision is a well-known depth-estimation method since it can generate dense and robust disparity maps. However, the real-time processing and low power dissipation, the specifications of the Internet-of-Thing (IoT) applications, are challenging for their computational complexity. In this paper, we propose a hardware-oriented SGM algorithm with pixel-level pipeline and region-optimized cost aggregation for high-speed processing and low hardware-resource usage. Firstly, the matching costs in a region are integrated with an optimization strategy to significantly reduce memory usage and improve the processing speed of the cost aggregation. Then, a two-layer parallel two-stage pipeline (TPTP) architecture, which enables pixel-level processing, is designed to calculate two directions (0° and 135°) aggregation to further solve the crucial computational bottleneck of the SGM algorithm. Finally, the architecture is demonstrated on a low-cost XILINX Spartan-7 device and an advanced Stratix-V FPGA device for VGA ( $640\times 480$ ) depth estimation. The experimental results show that the proposed architecture with compact hardware architecture also ensures accuracy. The pixel-level pipeline architecture enables a processing speed of 355 frames per second (fps) at 109MHz on the Spartan-7 FPGA device and 508 fps at 156MHz on the Stratix-V FPGA. Besides, the coprocessor respectively achieves an energy efficiency of 4.74 nJ/pixel with a power dissipation of 517mW and 4.29nJ/pixel with a power dissipation of 669mW on these two FPGAs.

This paper presents a compact vector quantizer based on the self-organizing map (SOM), which can fulfill the data compression task for high-speed image sequence. In this vector quantizer, we solve the most severe computational demands in the codebook learning mode and the image encoding mode by a reconfigurable complete-binary-adder-tree (RCBAT), where the arithmetic units are thoroughly reused. In this way, the hardware efficiency of our proposed vector quantizer is greatly improved. In addition, by distributing the codebook into the multi-parallel processing sub-blocks, our design obtains a high compression speed successfully. Furthermore, a mechanism of partial vector-component storage (PVCS) is adopted to make the compression ratio adjustable. Finally, the proposed vector quantizer has been implemented on the field programmable gate array (FPGA). The experimental results indicate that it respectively achieves a compression speed of 500 frames/s and a million connections per second (MCPS) of 28,494 (compression ratio is 64) when working at 79.8 MHz. Besides, compared with the previous scheme, our proposed quantizer achieves a reduction of 8% in hardware usage and an increase of 33% in compression speed. This means the proposed quantizer is hardware-efficient and can be used for high-speed image compression.