The emergence of hardware accelerators has brought about several orders of magnitude improvement in the speed of the deep neural-network (DNN) inference. Among such DNN accelerators, the Google tensor processing unit (TPU) has transpired to be the best-in-class, offering more than 15x speedup over the contemporary GPUs. However, the rapid growth in several DNN workloads conspires to escalate the energy consumptions of the TPU-based data-centers. In order to restrict the energy consumption of TPUs, we propose GreenTPU--a low-power near-threshold (NTC) TPU design paradigm. To ensure a high inference accuracy at a low-voltage operation, GreenTPU identifies the patterns in the error-causing activation sequences in the systolic array, and prevents further timing errors from similar patterns by intermittently boosting the operating voltage of the specific multiplier-and-accumulator units in the TPU. Compared to a cutting-edge timing error mitigation technique for TPUs, GreenTPU enables 2x to 3x higher performance (TOPS) in an NTC TPU, with a minimal loss in the prediction accuracy.
3 September 2021 in IEEE Transactions on Very Large Scale Integration (VLSI) Systems
Modern deep neural network (DNN) applications demand a remarkable processing throughput usually unmet by traditional Von Neumann architectures. Consequently, hardware accelerators, comprising a sea of multiplier-and-accumulate (MAC) units, have recently gained prominence in accelerating DNN inference engine. For example, tensor processing units (TPUs) account for a lion’s share of Google’s datacenter inference operations. The proliferation of real-time DNN predictions is accompanied by a tremendous energy budget. In quest of trimming the energy footprint of DNN accelerators, we propose Energy eFFicient and errOr Resilient TPU (EFFORT)—an energy optimized, yet high-performance TPU architecture, operating at the near-threshold computing (NTC) region. EFFORT promotes a better-than-worst case design by operating the NTC TPU at a substantially high frequency while keeping the voltage at the NTC nominal value. In order to tackle the timing errors due to such aggressive operation, we employ an opportunistic error mitigation strategy. In addition, we implement an in situ clock gating architecture, drastically reducing the MACs’ dynamic power consumption. Compared to a cutting-edge error mitigation technique for TPUs, EFFORT enables up to $2.5\times $ better performance at NTC with only 4% average accuracy drop across six out of eight DNN benchmarks.
AI evolution is accelerating and Deep Neural Network (DNN) inference accelerators are at the forefront of ad hoc architectures that are evolving to support the immense throughput required for AI computation. However, much more energy efficient design paradigms are inevitable to realize the complete potential of AI evolution and curtail energy consumption. The Near-Threshold Computing (NTC) design paradigm can serve as the best candidate for providing the required energy efficiency. However, NTC operation is plagued with ample performance and reliability concerns arising from the timing errors. In this paper, we dive deep into DNN architecture to uncover some unique challenges and opportunities for operation in the NTC paradigm. By performing rigorous simulations in TPU systolic array, we reveal the severity of timing errors and its impact on inference accuracy at NTC. We analyze various attributes—such as data–delay relationship, delay disparity within arithmetic units, utilization pattern, hardware homogeneity, workload characteristics—and uncover unique localized and global techniques to deal with the timing errors in NTC.
Increasing processing requirements in the Artificial Intelligence (AI) realm has led to the emergence of domain-specific architectures for Deep Neural Network (DNN) applications. Tensor Processing Unit (TPU), a DNN accelerator by Google, has emerged as a front runner outclassing its contemporaries, CPUs and GPUs, in performance by 15×–30×. TPUs have been deployed in Google data centers to cater to the performance demands. However, a TPU’s performance enhancement is accompanied by a mammoth power consumption. In the pursuit of lowering the energy utilization, this paper proposes PREDITOR—a low-power TPU operating in the Near-Threshold Computing (NTC) realm. PREDITOR uses mathematical analysis to mitigate the undetectable timing errors by boosting the voltage of the selective multiplier-and-accumulator units at specific intervals to enhance the performance of the NTC TPU, thereby ensuring a high inference accuracy at low voltage. PREDITOR offers up to 3×–5× improved performance in comparison to the leading-edge error mitigation schemes with a minor loss in accuracy.
15 January 2018 in IEEE Transactions on Emerging Topics in Computing
In this paper, we investigate the emerging security threats at Near-Threshold Computing (NTC) that are poised to jeopardize the trustworthy operation of future low-power electronic devices. A substantial research effort over the last decade has bolstered energy efficient operation in low-power computing. However, innovation in low-power security has received only marginal attention, thwarting a ubiquitous adoption of critical Internet of Things applications, such as wearable gadgets. Using a cross-layer methodology, we demonstrate that the timing fault vulnerability of a circuit rapidly increases as the operating conditions of the transistor devices shift from super-threshold to near-threshold values. Exploiting this vulnerability, we propose a novel threat model for NTC, referred to as a Timing Fault Attack at NTC (TITAN). TITAN relies on a malicious application software to induce timing fault attacks in the underlying NTC hardware. We evaluate the efficacy of TITAN using real hardware. Additionally, we propose two security parameters that dictate the fault resilience of a system. Based on those parameters, we show a 1.6X and a 2.8X deterioration in the fault resilience of a low-power operation, over a traditional super-threshold operation. Using a GDB driven analysis, we also present different user-level impacts of TITAN, on some real-life applications.
In this paper, we explore the emerging security threats of Near-Threshold computing (NTC). Researchers have shown that the delay sensitivity of a circuit to supply voltage variation tremendously increases, as the circuit’s operating conditions shift from traditional super-threshold values to NTC values. As a result, NTC systems become extremely vulnerable to timing fault attacks, jeopardizing trustworthy computing. Inspired by the operation of a polymorphic virus, we propose a novel threat model for NTC, referred to as a Focally Induced Fault Attack (FIFA). FIFA employs a machine learning framework to ascertain the circuit vulnerabilities and generates targeted software modules to cause a breach of end-user privacy. Our experimental results, obtained from a rigorous machine learning approach, indicate the efficacy of FIFA, in a low-power mobile platform.