Convolutional neural networks (CNN) have been extensively employed for image classification due to their high accuracy. However, inference is a computationally-intensive process that often requires hardware acceleration to operate in real time. For mobile devices, the power consumption of graphics processors (GPUs) is frequently prohibitive, and field-programmable gate arrays (FPGA) become a solution to perform inference at high speed. Although previous works have implemented CNN inference on FPGAs, their high utilization of on-chip memory and arithmetic resources complicate their application on resource-constrained edge devices. In this paper, we present a scalable, low power, low resource-utilization accelerator architecture for inference on the MobileNet V2 CNN. The architecture uses a heterogeneous system with an embedded processor as the main controller, external memory to store network data, and dedicated hardware implemented on reconfigurable logic with a scalable number of processing elements (PE). Implemented on a XCZU7EV FPGA running at 200 MHz and using four PEs, the accelerator infers with 87% top-5 accuracy and processes an image of pixels in 220 ms. It consumes 7.35 W of power and uses less than 30% of the logic and arithmetic resources used by other MobileNet FPGA accelerators.
This paper presents the architecture of a hardware accelerator for a cellular neural network (CeNN) with an application to real-time edge detection on visible-range and infrared video. The accelerator features fully-pipelined processing elements (PEs) that exploit the data parallelism in the algorithm to perform an iteration of the CeNN on a stream of video data with high throughput. The memory architecture exploits the locality of reference in the CeNN, so that each PE uses only 5 line buffers to store pixel, state, and output data, thus achieving low on-chip memory utilization. Implemented on a Xilinx XC7A200T FPGA running at 245MHz, the accelerator performs edge detection on 1080p video using a single CeNN iteration with a throughput of 118 frames per second (fps), a total latency of 15.7us, and 618mW of power consumption. The architecture features static reconfiguration to store built-in kernels and to add more PEs to support multiple iterations of the CeNN algorithm. More kernels can be added dynamically through a serial interface.