The You Only Look Once (YOLO) neural network has great advantages and extensive applications in computer vision. The convolutional layers are the most important part of the neural network and take up most of the computation time. Improving the efficiency of the convolution operations can greatly increase the speed of the neural network. Field programmable gate arrays (FPGAs) have been widely used in accelerators for convolutional neural networks (CNNs) thanks to their configurability and parallel computing. This paper proposes a design space exploration for the YOLO neural network based on FPGA. A data block transmission strategy is proposed and a multiply and accumulate (MAC) design, which consists of two 14 × 14 processing element (PE) matrices, is designed. The PE matrices are configurable for different CNNs according to the given required functions. In order to take full advantage of the limited logical resources and the memory bandwidth on the given FPGA device and to simultaneously achieve the best performance, an improved roofline model is used to evaluate the hardware design to balance the computing throughput and the memory bandwidth requirement. The accelerator achieves 41.99 giga operations per second (GOPS) and consumes 7.50 W running at the frequency of 100 MHz on the Xilinx ZC706 board.

Convolutional neural networks (CNNs) have been widely applied in the field of computer vision due to their inherent advantages in image feature extraction. However, it is difficult to implement CNNs directly on embedded platforms owing to excessive calculations of CNNs. Field Programmable Gate Arrays have been popular in CNN accelerators because of their configurability and high energy efficiency. Given the highly parallel workloads of the CNN, a CNN accelerator with a 14 × 16 processing element array is designed in this study to accelerate the CNN inference. Besides, a loop tiling strategy for convolutional layers is proposed to efficiently transmit feature maps. Additionally, the roofline model is employed to explore the best tiling parameters for optimal performance. Finally, the accelerator written in Verilog‐HDL language is implemented on the Xilinx Zynq‐7045 evaluation platform. At an operating frequency of 200 MHz, the proposed accelerator can achieve a performance of 57.24 giga operations per second on You Only Look Once v2‐tiny and 78.39 GOPS on Visual Geometry Group‐16. The accelerator only consumes 224 DSPs, demonstrating a better performance compared with the previous works.

Deep convolutional neural networks (DNNs) have been widely used in many applications, particularly in machine vision. It is challenging to accelerate DNNs on embedded systems because real-world machine vision applications should reserve a lot of external memory bandwidth for other tasks, such as video capture and display, while leaving little bandwidth for accelerating DNNs. In order to solve this issue, in this study, we propose a high-throughput accelerator, called reconfigurable tiny neural network accelerator (ReTiNNA), for the bandwidth-limited system and present a real-time object detection system for the high-resolution video image. We first present a dedicated computation engine that takes different data mapping methods for various filter types to improve data reuse and reduce hardware resources. We then propose an adaptive layer-wise tiling strategy that tiles the feature maps into strips to reduce the control complexity of data transmission dramatically and to improve the efficiency of data transmission. Finally, a design space exploration (DSE) approach is presented to explore design space more accurately in the case of insufficient bandwidth to improve the performance of the low-bandwidth accelerator. With a low bandwidth of 2.23 GB/s and a low hardware consumption of 90.261K LUTs and 448 DSPs, ReTiNNA can still achieve a high performance of 155.86 GOPS on VGG16 and 68.20 GOPS on ResNet50, which is better than other state-of-the-art designs implemented on FPGA devices. Furthermore, the real-time object detection system can achieve a high object detection speed of 19 fps for high-resolution video.

Recent researches on deep convolution neural networks have proposed some compact networks, such as MobileNet, but its main computation, depthwise separable convolution (DWC), which reduces the reusable data and improves the requirement of data loading efficiency. Although DWC can effectively reduce the amount of network computation, it needs a special accelerator to enhance the inference speed. This paper proposes a high‐performance accelerator for DWC based on the commonly used acceleration platform field‐programmable gate array. The proposed accelerator supports the computation of both standard convolutions (SCs) and DWC as well as two activation functions. In addition, two data storage formats are used to maintain the data loading efficiency for different input requirements of SC and DWC under high parallelism. Furthermore, a processing unit that can execute two 8 × 8‐bit multiplications inside one digital signal processor (DSP) is designed to make the best use of the DSP hardware resources. Finally, the accelerator is implemented on ZYNQ ZC706 at 200 MHz. Consuming only 392 DSPs, the accelerator achieves 134.5 giga operations per second (GOPS) and 209.4 frames per second (FPS) on MobileNet V1 as well as 96.4 GOPS and 250.4 FPS on MobileNet V2. Experimental results demonstrate that this design provides a better DSP efficiency than previous works.

In the field of hardware accelerators for convolutional neural network (CNN) inference, quantization techniques have been widely employed to enhance the performance. The prevailing quantization scheme of the accelerator at present is using signed 8‐bit integer variables (INT8). CNN accelerators support INT8, while lower precision INT4 is less common. Accelerators supporting INT4 depthwise separable convolution (DWC) are even rarer. Therefore, this article presents a high‐performance CNN accelerator that not only supports 8‐bit and 4‐bit data but also supports standard convolution (SC) and DWC. Additionally, in order to improve the transmission efficiency of DWC, an intermediate cache strategy is proposed, using a pointwise convolution (PW) input buffer (PW BUF) to store output data from depthwise convolution (DW) to avoid off‐chip transmission. Furthermore, to address the issue of a DSP cannot perform two 4 × 4‐bit multiplications when dealing with DW, a processing element (PE) is designed to make full use of DSP hardware resources. Finally, this accelerator is implemented on ZYNQ ZC706 with a frequency of 200 MHz. Experimental results show that it achieves a performance up to 307.88 giga operations per second (GOPS) on VGG, reaching 97.9% peak performance; while on MobileNet, it achieves efficient performance with 206.43 GOPS with only 392 DSPs. Compared with mainstream CNN accelerators, it increases DSP utilization rate (GOPS/DSP) by 1.5× to 33.5×.

Currently, there are two mainstream acceleration methods; one is mixed precision and the other is sparsity. Few accelerators support both mixed precision and sparsity, and most enable precision configurations across layers rather than within a single layer. Furthermore, most of accelerators adopt the traditional two’s complement (2C) data representation method, and we found that 2C brings many invalid “1” when representing signed data, which brings more resources overhead for mixed precision and many invalid operations for bit-level sparsity. Therefore, we propose a high-efficiency accelerator featuring a precision-scalable Sign-Magnitude Processing Element (SM-PE), which adopts a data representation method of SM and can flexibly support various precision calculations (2, 4, 8 bits) and bit-level sparsity. In addition, a dynamic quantization algorithm named DoReFaLike and a bit-level column sparsity (BLCS) technique are proposed to improve the efficiency of SM-PEs. Under the same accuracy constraint, the sparsity rate of the SM scheme is 3.5× higher than that of the 2C format. The accelerator has been synthesized on a 55nm CMOS ASIC platform. When scaled to 28nm, experimental results show that the energy efficiency of the proposed accelerator reaches 15.50, 25.37, 101.54 TOPS/W with 8-bit, 4-bit, and 2-bit input activations, respectively, and weights represented in sparse 8-bit precision, operating at 400 MHz. Compared to state-of-the-art accelerators, the proposed design achieves a performance improvement of 1.1× to 3.9×.