19 March 2021 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Deep neural network (DNN)-based object detection has been investigated and applied to various real-time applications. However, it is hard to employ the DNNs in embedded systems due to their high computational complexity and deep-layered structure. Although several field-programmable gate array (FPGA) implementations have been presented recently for real-time object detection, they suffer from either low throughput or low detection accuracy. In this article, we propose an efficient computing system for real-time SSDLite object detection on FPGA devices, which includes novel hardware architecture and system optimization techniques. In the proposed hardware architecture, a neural processing unit (NPU) that consists of heterogeneous units, such as band processing, scaling, and accumulating, and data fetching and formatting units is designed to accelerate the DNNs efficiently. In addition, system optimization techniques are presented to improve the throughput further. A task control unit is employed to balance the workload and increase the utilization of heterogeneous units in the NPU, and the object detection algorithm is refined accordingly. The proposed architecture is realized on an Intel Arria 10 FPGA and enhances the throughput by up to 13.6× compared to the state-of-the-art FPGA implementation.
12 November 2019 in IEEE Transactions on Circuits and Systems I: Regular Papers
This article presents an approximate signed digit representation (ASD) which quantizes the weights of convolutional neural networks (CNNs) in order to make multiplier-less CNNs without performing any retraining process. Unlike the existing methods that necessitate retraining for weight quantization, the proposed method directly converts full-precision weights of CNN models into low-precision ones, attaining accuracy comparable to that of full-precision models on the Image classification tasks without going through retraining. Therefore, it is effective in saving the retraining time as well as the related computational cost. As the proposed method simplifies the weights to have up to two non-zero digits, multiplication can be realized with only add and shift operations, resulting in a speed-up of inference time and a reduction of energy consumption and hardware complexity. Experiments conducted for famous CNN architectures, such as AlexNet, VGG-16, ResNet-18 and SqueezeNet, show that the proposed method reduces the model size by 73% at the cost of a little increase of error rate, which ranges from 0.09% to 1.5% on ImageNet dataset. Compared to the previous architecture built with multipliers, the proposed multiplier-less convolution architecture reduces the critical-path delay by 52% and mitigates the hardware complexity and power consumption by more than 50%.