Systolic arrays (SAs) are highly parallel pipelined structures capable of executing various tasks such as matrix multiplication and convolution. They comprise a grid of usually homogeneous processing units (PUs) that are responsible for the multiply-accumulate (MAC) operations in the case of matrix multiplication. It is not rare for a PU input to be zero-valued, in which case the PU becomes idle and the array becomes underutilized. In this paper we consider a solution to employ the underutilized PUs via simultaneous multithreading (SMT). We explore the design space of a SMT-SA variant and evaluate its performance, area efficiency, and energy consumption. In addition, we suggest a tiling method to reduce area overheads. Our evaluation shows that a 4-thread FP16-based SMT-SA achieves speedups of up to 3.6x as compared to conventional SA, with 1.7x area overhead and negligible energy overhead.

Deep neural networks (DNNs) excel in various applications, such as computer vision, natural language processing, and other mission-critical systems. As the computational complexity of these models grows, there is an increasing need for specialized accelerators to handle the demanding workloads. In response, advancements in Very Large Scale Integration (VLSI) process nodes have significantly intensified the development of machine learning (ML) accelerators, offering enhanced transistor miniaturization and power efficiency. However, the susceptibility of these advanced nodes to transistor aging poses risks to ML accelerator performance, prediction accuracy, and reliability, which can impact the functional safety of mission-critical systems. This study focuses on the impact of asymmetric transistor aging, induced by Bias Temperature Instability (BTI), on systolic arrays (SAs), which are integral to many ML accelerators in mission-critical systems. Our aging-aware analysis indicates that SAs experience asymmetric aging, causing logical elements to age at varying rates. In addition, our simulations show that asymmetric transistor aging introduces persistent and transient faults in the SA’s datapath, compromising the overall resiliency of the ML model. Our simulation results show that even with less than 1% of transient failure events, the top-1 prediction accuracy of ResNet-18 ML model drops significantly by 32–50% and with approximately 0.8% of transient failure events PTQ4ViT drops by almost 90%. To address this issue, we propose new hardware mechanisms and design flow solutions that can successfully mitigate the impact of asymmetric transistor aging on ML accelerator reliability with minimal power and area overhead.

Convolutional Neural Networks (CNNs) are broadly used in numerous applications such as computer vision and image classification. Although CNN models deliver state-of-the-art accuracy, they require heavy computational resources that are not always affordable or available on every platform. Limited performance, system cost, and energy consumption, such as in edge devices, argue for the optimization of computations in neural networks. Toward this end, we propose herein the value-locality-based compression (VELCRO) algorithm for neural networks. VELCRO is a method to compress general-purpose neural networks that are deployed for a small subset of focused specialized tasks. Although this study focuses on CNNs, VELCRO can be used to compress any deep neural network. VELCRO relies on the property of value locality, which suggests that activation functions exhibit values in proximity through the inference process when the network is used for specialized tasks. VELCRO consists of two stages: a preprocessing stage that identifies output elements of the activation function with a high degree of value locality, and a compression stage that replaces these elements with their corresponding average arithmetic values. As a result, VELCRO not only saves the computation of the replaced activations but also avoids processing their corresponding output feature map elements. Unlike common neural network compression algorithms, which require computationally intensive training processes, VELCRO introduces significantly fewer computational requirements. An analysis of our experiments indicates that, when CNNs are used for specialized tasks, they introduce a high degree of value locality relative to the general-purpose case. In addition, the experimental results show that without any training process, VELCRO produces a compression-saving ratio in the range 13.5–30.0% with no degradation in accuracy. Finally, the experimental results indicate that, when VELCRO is used with a relatively low compression target, it significantly improves the accuracy by 2–20% for specialized CNN tasks.

Deep neural networks (DNNs) require abundant multiply-and-accumulate (MAC) operations. Thanks to DNNs' ability to accommodate noise, some of the computational burden is commonly mitigated by quantization – that is, by using lower precision floating-point operations. Layer granularity is the preferred method, as it is easily mapped to commodity hardware. In this paper, we propose Dynamic Asymmetric Architecture (DAA), in which the micro-architecture decides what the precision of each MAC operation should be during runtime. We demonstrate a DAA with two data streams and a value-based controller that decides which data stream deserves the higher precision resource. We evaluate this mechanism in terms of accuracy on a number of convolutional neural networks (CNNs) and demonstrate its feasibility on top of a systolic array. Our experimental analysis shows that DAA potentially achieves 2x throughput improvement for ResNet-18 while saving 35% of the energy with less than 0.5% degradation in accuracy.