This paper presents an end-to-end object-detection accelerator that processes raw Bayer images to generate detection results. The accelerator utilizes histogram of oriented gradients (HOG) features in combination with a support vector machine (SVM) classifier. The proposed HOG for raw images (HOGR) skips the image signal processors which consume a significant amount of power (about 2.5X of an accelerator). The proposed architecture temporally partitions the algorithms and time multiplexes the logic such that the accelerator works on the same frequency with the image sensor using affordable resources. The prototype is verified using 1080p ( $1920\times 1080$ ) and VGA ( $640\times 480$ ) raw videos on Altera Arria10 and Cyclone IV field programmable gate array (FPGA) platforms. The accelerator can process 1080p raw videos with a 12-scale pyramid at 60 frames per second (FPS) under the pixel frequency of corresponding image sensor (148.5 MHz), consuming 510 Kbit block memory. To the best knowledge of the authors, this is the first end-to-end HOG+SVM accelerator that takes raw Bayer images as input, skips the ISP pipeline for resource optimization from the system perspective, and is synchronized with the image sensor.

The Hough Transform (HT) is a method for extracting straight lines from an edge image. The main limitations of the HT for usage in actual applications are computation time and storage requirements. This paper reports a hardware architecture for HT implementation on a Field Programmable Gate Array (FPGA) with parallelized voting procedure. The 2-dimensional accumulator array, namely the Hough space in parametric form (ρ, θ), for computing the strength of each line by a voting mechanism is mapped on a 1-dimensional array with regular increments of θ. Then, this Hough space is divided into a number of parallel parts. The computation of (ρ, θ) for the edge pixels and the voting procedure for straight-line determination are therefore executable in parallel. In addition, a synchronized initialization for the Hough space further increases the speed of straight-line detection, so that XGA video processing becomes possible. The designed prototype system has been synthesized on a DE4 platform with a Stratix-IV FPGA device. In the application of road-lane detection, the average processing speed of this HT implementation is 5.4ms per XGA-frame at 200 MHz working frequency.

This paper presents a compact vector quantizer based on the self-organizing map (SOM), which can fulfill the data compression task for high-speed image sequence. In this vector quantizer, we solve the most severe computational demands in the codebook learning mode and the image encoding mode by a reconfigurable complete-binary-adder-tree (RCBAT), where the arithmetic units are thoroughly reused. In this way, the hardware efficiency of our proposed vector quantizer is greatly improved. In addition, by distributing the codebook into the multi-parallel processing sub-blocks, our design obtains a high compression speed successfully. Furthermore, a mechanism of partial vector-component storage (PVCS) is adopted to make the compression ratio adjustable. Finally, the proposed vector quantizer has been implemented on the field programmable gate array (FPGA). The experimental results indicate that it respectively achieves a compression speed of 500 frames/s and a million connections per second (MCPS) of 28,494 (compression ratio is 64) when working at 79.8 MHz. Besides, compared with the previous scheme, our proposed quantizer achieves a reduction of 8% in hardware usage and an increase of 33% in compression speed. This means the proposed quantizer is hardware-efficient and can be used for high-speed image compression.

Many computer-vision and machine-learning applications in robotics, mobile, wearable devices, and automotive domains are constrained by their real-time performance requirements. This paper reports a dual-feature-based object recognition coprocessor that exploits both histogram of oriented gradient (HOG) and Haar-like descriptors with a cell-based parallel sliding-window recognition mechanism. The feature extraction circuitry for HOG and Haar-like descriptors is implemented by a pixel-based pipelined architecture, which synchronizes to the pixel frequency from the image sensor. After extracting each cell feature vector, a cell-based sliding window scheme enables parallelized recognition for all windows, which contain this cell. The nearest neighbor search classifier is, respectively, applied to the HOG and Haar-like feature space. The complementary aspects of the two feature domains enable a hardware-friendly implementation of the binary classification for pedestrian detection with improved accuracy. A proof-of-concept prototype chip fabricated in a 65-nm SOI CMOS, having thin gate oxide and buried oxide layers (SOTB CMOS), with 3.22-mm ² core area achieves an energy efficiency of 1.52 nJ/pixel and a processing speed of 30 fps for 1024 × 1616-pixel image frames at 200-MHz recognition working frequency and 1-V supply voltage. Furthermore, multiple chips can implement image scaling, since the designed chip has image-size flexibility attributable to the pixel-based architecture.

As an alternative to conventional single-instruction-multiple-data (SIMD) mode solutions with massive parallelism for self-organizing-map (SOM) neural network models, this paper reports a memory-based proposal for the learning vector quantization (LVQ), which is a variant of SOM. A dual-mode LVQ system, enabling both on-chip learning and classification, is implemented by using a reconfigurable pipeline with parallel p-word input (R-PPPI) architecture. As a consequence of the reuse of R-PPPI for solving the most severe computational demands in both modes, power dissipation and Si-area consumption can be dramatically reduced in comparison to previous LVQ implementations. In addition, the designed LVQ ASIC has high flexibility with respect to feature-vector dimensionality and reference-vector number, allowing the execution of many different machine-learning applications. The fabricated test chip in 180 nm CMOS with parallel 8-word inputs and 102 K-bit on-chip memory achieves low power consumption of 66.38 mW (at 75 MHz and 1.8 V) and high learning speed of clock cycles per d-dimensional sample vector where R is the reference-vector number.

Learning vector quantization (LVQ) neural networks have already been successfully applied for image compression and object recognition. In this paper, we propose a modular and reconfigurable pipeline architecture (MRPA) for LVQ. The MRPA consists of dynamically reconfigurable modules and realizes a run-time and on-chip configuration for recognition and learning. Prototype fabrication in 65-nm CMOS technology verifies high integration density and memory-utilization efficiency, good performance, and considerable flexibility in vector dimensionality, number of weight-vectors, and adaption strategies. Compared with the embedded microprocessors, which rely on single-instruction-multiple-data processing, the developed prototype increases the performance of both recognition and learning operations. The MRPA prototype shows improvements by factors of approximately 40 and 101 on the well-established performance metrics million connections per second for recognition and million connection updates per second for learning, respectively.