Smartphones, wearables, and Internet of Things (IoT) devices have already revolutionized many aspects of our everyday lives, enabling ubiquitously connected machine-to-machine, machine-to-human, and human-to-human communication towards the realization of smarter environments in our homes and across the domains of medical care, transportation, energy grids, industry automation, and defense. This article surveys the landscape of energy management solutions for mobile and IoT devices, to maximize performance and quality-of-service within the varying resource-constraints of these devices.

The complexity of heterogeneous mobile platforms is growing at a rate faster than our ability to manage them optimally at runtime. For example, state-of-the-art systems-on-chip (SoCs) enable controlling the type (Big/Little), number, and frequency of active cores. Managing these platforms becomes challenging with the increase in the type, number, and supported frequency levels of the cores. However, existing solutions used in mobile platforms still rely on simple heuristics based on the utilization of cores. This paper presents a novel and practical imitation learning (IL) framework for dynamically controlling the type (Big/Little), number, and the frequencies of active cores in heterogeneous mobile processors. We present efficient approaches for constructing an Oracle policy to optimize different objective functions, such as energy and performance per Watt (PPW). The Oracle policies enable us to design low-overhead power management policies that achieve near-optimal performance matching the Oracle. Experiments on a commercial platform with 19 benchmarks show on an average 101% PPW improvement compared to the default interactive governor.

In-memory computing reduces latency and energy consumption of Deep Neural Networks (DNNs) by reducing the number of off-chip memory accesses. However, crossbar-based in-memory computing may significantly increase the volume of on-chip communication since the weights and activations are on-chip. State-of-the-art interconnect methodologies for in-memory computing deploy a bus-based network or mesh-based Network-on-Chip (NoC). Our experiments show that up to 90% of the total inference latency of a DNN hardware is spent on on-chip communication when the bus-based network is used. To reduce the communication latency, we propose a methodology to generate an NoC architecture along with a scheduling technique customized for different DNNs. We prove mathematically that the generated NoC architecture and corresponding schedules achieve the minimum possible communication latency for a given DNN. Furthermore, we generalize the proposed solution for edge computing and cloud computing. Experimental evaluations on a wide range of DNNs show that the proposed NoC architecture enables 20%-80% reduction in communication latency with respect to state-of-the-art interconnect solutions.

State-of-the-art in-memory computing (IMC) architectures employ an array of homogeneous tiles and severely underutilize processing elements (PEs). In this article, the authors propose an area and energy optimization methodology to generate a heterogeneous IMC architecture coupled with an optimized Network-on-Chip (NoC) for deep neural network (DNN) acceleration. – Yiran Chen, Duke University