Compiler frameworks are crucial for the widespread use of FPGA-based deep learning accelerators. They allow researchers and developers, who are not familiar with hardware engineering, to harness the performance attained by domain-specific logic. There exists a variety of frameworks for conventional artificial neural networks. However, not much research effort has been put into the creation of frameworks optimized for spiking neural networks (SNNs). This new generation of neural networks becomes increasingly interesting for the deployment of AI on edge devices, which have tight power and resource constraints. Our end-to-end framework E3NE automates the generation of efficient SNN inference logic for FPGAs. Based on a PyTorch model and user parameters, it applies various optimizations and assesses trade-offs inherent to spike-based accelerators. Multiple levels of parallelism and the use of an emerging neural encoding scheme result in an efficiency superior to previous SNN hardware implementations. For a similar model, E3NE uses less than 50% of hardware resources and 20% less power, while reducing the latency by an order of magnitude. Furthermore, scalability and generality allowed the deployment of the large-scale SNN models AlexNet and VGG.

Spiking neural networks (SNNs) have advantages in latency and energy efficiency over traditional artificial neural networks (ANNs) due to their event-driven computation mechanism and the replacement of energy-consuming weight multiplication with addition. However, to achieve high accuracy, it usually requires long spike trains to ensure accuracy, usually more than 1000 time steps. This offsets the computation efficiency brought by SNNs because a longer spike train means a larger number of operations and larger latency. In this article, we propose a radix-encoded SNN, which has ultrashort spike trains. Specifically, it is able to use less than six time steps to achieve even higher accuracy than its traditional counterpart. We also develop a method to fit our radix encoding technique into the ANN-to-SNN conversion approach so that we can train radix-encoded SNNs more efficiently on mature platforms and hardware. Experiments show that our radix encoding can achieve 25 $\times$ improvement in latency and 1.7% improvement in accuracy compared to the state-of-the-art method using the VGG-16 network on the CIFAR-10 dataset.

Silicon photonic networks are revolutionizing computing systems by improving the energy efficiency, bandwidth, and latency of data movements. Optical modulators, such as microresonators and Mach-Zehnder Interferometers (MZIs), are the basic building blocks of silicon photonic networks. This work proposes a SPICE-compatible electro-optical co-simulation model, BOSIM, to systematically study optical modulators using PN, PIN, and metal-insulator-silicon (MIS) capacitor device technologies. BOSIM holistically models both transient and steady state properties, such as switching speed, power, transmission spectrum, area and carrier distribution. BOSIM is validated by the measured data from eight research groups and companies. Compared to microresonators, BOSIM shows MZIs are fast, with a high extinction ratio and large bandwidth but in the sacrifice of loss, energy, and area. Using a PIN diode over a PN diode can save area, but retain the loss and energy, while an MIS capacitor has the shock response of carrier distribution in a narrow range and is marginalized gradually. For instance, an MZI can achieve a 2.5X bit rate, 6.06X extinction ratio, 71.04X 3dB bandwidth, but costs at least 1.93X passing loss, 1.46X energy consumption and 16.67X area, compared with microresonator.

While many new applications prefer manycore processor with a large number of cores, the exploding communications among multiple cores, caches, and off-chip memories is posing a fundamental challenge on manycore designs. Silicon photonics based interconnection network promises high bandwidth, low latency, and high energy efficiency, and can potentially meet the communication requirements of manycore processors. In this work, we propose CAMON, a small low-cost silicon photonic chiplet integrated into the manycore processor package. CAMON chiplet can effectively alleviate the communication bottlenecks of manycore processors and improve the energy efficiency of data movement, especially for large-scale systems. We develop a distributed arbitration system, a low-power low-latency optical interface, and an off-chip laser preactivation mechanism for CAMON. Experiment results show that compared with the electrical network, CAMON can improve the full-system performance per energy by 4.6X, speed up the manycore processor by 2.7X, and save the area of the processor die by 3%, in a 512-core system.

Edge devices demand low energy consumption, cost, and small form factor. To efficiently deploy convolutional neural network (CNN) models on the edge device, energy-aware model compression becomes extremely important. However, existing work did not study this problem well because of the lack of considering the diversity of dataflow types in hardware architectures. In this article, we propose EDCompress (EDC), an energy-aware model compression method for various dataflows. It can effectively reduce the energy consumption of various edge devices, with different dataflow types. Considering the very nature of model compression procedures, we recast the optimization process to a multistep problem and solve it by reinforcement learning algorithms. We also propose a multidimensional multistep (MDMS) optimization method, which shows higher compressing capability than the traditional multistep method. Experiments show that EDC could improve 20x, 17x, and 26x energy efficiency in VGG-16, MobileNet, and LeNet-5 networks, respectively, with negligible loss of accuracy. EDC could also indicate the optimal dataflow type for specific neural networks in terms of energy consumption, which can guide the deployment of CNN on hardware.

Large language models (LLMs) have garnered substantial attention due to their promising applications in diverse domains. Nevertheless, the increasing size of LLMs comes with a significant surge in the computational requirements for training and deployment. Memristor crossbars have emerged as a promising solution, which demonstrated a small footprint and remarkably high energy efficiency in computer vision (CV) models. Memristors possess higher density compared to conventional memory technologies, making them highly suitable for effectively managing the extreme model size associated with LLMs. However, deploying LLMs on memristor crossbars faces three major challenges. First, the size of LLMs increases rapidly, already surpassing the capabilities of state-of-the-art memristor chips. Second, LLMs often incorporate multi-head attention blocks, which involve non-weight stationary multiplications that traditional memristor crossbars cannot support. Third, while memristor crossbars excel at performing linear operations, they are not capable of executing complex nonlinear operations in LLM such as softmax and layer normalization. To address these challenges, we present a novel architecture for the memristor crossbar that enables the deployment of state-of-the-art LLM on a single chip or package, eliminating the energy and time inefficiencies associated with off-chip communication. Our testing on BERT$_{\text {Large}}$ showed negligible accuracy loss. Compared to traditional memristor crossbars, our architecture achieves enhancements of up to $39\times$ in area overhead and $18\times$ in energy consumption. Compared to modern TPU/GPU systems, our architecture demonstrates at least a $68\times$ reduction in the area-delay product and a significant 69% energy consumption reduction.