16 October 2018 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
We propose a novel memory architecture for in-memory computation called McDRAM, where DRAM dies are equipped with a large number of multiply accumulate (MAC) units to perform matrix computation for neural networks. By exploiting high internal memory bandwidth and reducing off-chip memory accesses, McDRAM realizes both low latency and energy efficient computation. In our experiments, we obtained the chip layout based on the state-of-the-art memory, LPDDR4 where McDRAM is equipped with 2048 MACs in a single chip package with a small area overhead (4.7%). Compared with the state-of-the-art accelerator, TPU and the power-efficient GPU, Nvidia P4, McDRAM offers $9.5{\times }$ and $14.4{\times }$ speedup, respectively, in the case that the large-scale MLPs and RNNs adopt the batch size of 1. McDRAM also gives $2.1{\times }$ and $3.7{\times }$ better computational efficiency in TOPS/W than TPU and P4, respectively, for the large batches.
18 August 2017 in IEEE Design & Test
Accelerating convolutional neural networks (CNNs) is becoming more and more important due to their ever-in-creasing range of applications, e.g., image and video classification in data centers. Based on the observations that the significant amount of weights and activations in CNNs can be zero without losing quality, we propose a novel zero-aware hardware accelerator for CNNs named ZeNA, which exploits both zero weights and activations. Moreover, we report zero-induced load imbalance problems at two levels of scale in a zero-aware parallel CNN accelerator and present zero-aware kernel allocation and dynamic work group allocation as solutions. According to our experiments with a cycle-accurate simulation model, RTL, and layout design, ZeNA achieves 4.4×/2× (AlexNet) and 5.6×/2.4× (VGG-16) speedup compared to an existing state-of-the-art zero-agnostic/zero-activation-aware accelerator.
22 July 2020 in IEEE Access
The energy efficiency of accelerating hundreds of MB-large deep neural networks (DNNs) in a mobile environment is less than that of a server-class big chip accelerator because of the limited power budget, silicon area, and smaller buffer size of static random access memory associated with mobile systems. To address this challenge and provide powerful computing capability for processing large DNN models in power/resource-limited mobile systems, we propose McDRAM v2, which is a novel in-dynamic random access memory (DRAM) systolic array accelerator architecture. McDRAM v2 makes the best use of large in-DRAM bandwidths for accelerating various DNN applications. It can handle large DNN models without off-chip memory accesses, in a fast and efficient manner, by exposing the large DRAM capacity and large in-DRAM bandwidth directly to an input systolic array of a processing element matrix. Additionally, it maximizes data reuse using a systolic multiply–accumulate (MAC) structure. The proposed architecture maximizes the utilization of large-scale MAC units by judiciously exploiting the DRAM's internal bus and buffer structure. An evaluation of large DNN models in the fields of image classification, natural language processing, and recommendation systems shows that it achieves 1.7 times tera operations per second (TOPS), 3.7 times TOPS/watt, and 8.6 times TOPS/mm2 improvements over a state-of-the–art mobile graphics processing unit accelerator, and 4.1 times better energy efficiency over a state-of-the-art server-class accelerator. Moreover, it incurs a minimal overhead, i.e., a 9.7% increase in area, and uses less than 4.4 W of peak operating power.
20 April 2021 in IEEE Access
Few-shot learning deals with a small amount of data which incurs insufficient performance with conventional cross-entropy loss. We propose a pretraining approach for few-shot learning scenarios. That is, considering that the feature extractor quality is a critical factor in few-shot learning, we augment the feature extractor using a contrastive learning technique. It is reported that supervised contrastive learning applied to base class training in transductive few-shot training pipeline leads to improved results, outperforming the state-of-the-art methods on Mini-ImageNet and CUB. Furthermore, our experiment shows that a much larger dataset is needed to retain few-shot classification accuracy when domain-shift degradation exists, and if our method is applied, the need for a large dataset is eliminated. The accuracy gain can be translated to a runtime reduction of 3.87× in a resource-constrained environment.