Tiny machine learning (TinyML) has become an emerging field according to the rapid growth in the area of the internet of things (IoT). However, most deep learning algorithms are too complex, require a lot of memory to store data, and consume an enormous amount of energy for calculation/data movement; therefore, the algorithms are not suitable for IoT devices such as various sensors and imaging systems. Furthermore, typical hardware accelerators cannot be embedded in these resource-constrained edge devices, and they are difficult to drive real-time inference processing as well. To perform the real-time processing on these battery-operated devices, deep learning models should be compact and hardware-optimized, and hardware accelerator designs also have to be lightweight and consume extremely low energy. Therefore, we present an optimized network model through model simplification and compression for the hardware to be implemented, and propose a hardware architecture for a lightweight and energy-efficient deep learning accelerator. The experimental results demonstrate that our optimized model successfully performs object detection, and the proposed hardware design achieves 1.25× and 4.27× smaller logic and BRAM size, respectively, and its energy consumption is approximately 10.37× lower than previous similar works with 43.95 fps as a real-time process under an operating frequency of 100 MHz on a Xilinx ZC702 FPGA.
Though custom deep learning (DL) hardware accelerators are attractive for making inferences in edge computing devices, their design and implementation remain a challenge. Open-source frameworks exist for exploring DL hardware accelerators. Gemmini is an open-source systolic array generator for agile DL accelerator exploration. This paper details the hardware/software components generated using Gemmini. The general matrix-to-matrix multiplication (GEMM) of different dataflow options, including output/weight stationary (OS/WS), was explored in Gemmini to estimate the performance relative to a CPU implementation. The Gemmini hardware was implemented on an FPGA device to explore the effect of several accelerator parameters, including array size, memory capacity, and the CPU/hardware image-to-column (im2col) module, on metrics such as the area, frequency, and power. This work revealed that regarding the performance, the WS dataflow offered a speedup of 3× relative to the OS dataflow, and the hardware im2col operation offered a speedup of 1.1× relative to the operation on the CPU. For hardware resources, an increase in the array size by a factor of 2 led to an increase in both the area and power by a factor of 3.3, and the im2col module led to an increase in area and power by factors of 1.01 and 1.06, respectively.
The high dynamic range (HDR) has become very important because of the rapid increase in demand for a variety of applications. However, most of them were implemented by expensive systems due to the high complex computation for processing the real-time 4K UHD video. In the proposed hardware, the non-linear camera response function (CRF) with the area optimization of logarithmic computations has been applied to improve HDR quality. And, for embedding in Field Programmable Gate Array (FPGA), we implement a dedicated hardware using 4006 lookup table (LUT) and 21KB sized internal memory. The proposed architecture enables a real-time HDR processing with pipelining for a UHD video (8 Mega pixels) at 30 frames per second.
This paper presents novel hardware architecture with low-complexity color conversion scheme and parallel processing of red region detection for the applications of automatic traffic sign detection system. By the inherent parallelism of the various red region detections, we designed a fully pipelined architecture implemented on the FPGA platform. The proposed architecture enables a real-time traffic sign recognition (TSR) processing for a full-HD video (2 Mega pixels) at 60 frames per second.
Near-data processing (NDP), a solution to reduce data movement overhead between host and memory, should not interfere with host access to ensure system fairness. We propose a cost-effective and energy-efficient LRDIMM-based NDP architecture (HAIL-DIMM) that can seamlessly interleave NDP and host access and is a drop-in replacement for existing main memory modules. The proposed NDP exploits the interleaving capability of the memory controller to interleave NDP and host access naturally. To take advantage of bank interleaving, an atomic operation of the proposed NDP, which consists of data movement and computation, is recognized by the memory controller as a DDR READ/WRITE but by the HAIL-DIMM as NDP based on the request's address. We implement a prototype of the proposed NDP architecture on an FPGA platform as proof of concept. The evaluation results show that the NDP system achieves up to 2.19x speedup in latency and up to 45.4% energy saving for data movement over the baseline system for memory-bound workloads.
Deep learning networks can be accelerated by reducing the overall network volume using quantization or pruning techniques. The well-known quantization technique is Post Training Quantization (PTQ) and Quantization Aware Training (QAT). We applied an INT8 quantized network to design deep learning acceleration hardware and found that the performance of the deep learning network deteriorated due to errors occurring in the mult/shift based re-quantization step. This quantization error becomes a bigger problem in the training process rather than inference, and the FP32 arithmetic operator is applied to prevent the resulting accuracy drop. In this paper, we investigate whether the use of FP32 operators can outperform employing mult/shift operators under specific conditions. We accomplish this by analyzing the data flow based on output channel tiling and conducting a size analysis of the implemented hardware.