Transformer-based models have achieved remarkable success across various AI tasks, but their growing complexity has led to significant computational and memory demands. While most optimization efforts have focused on linear operations such as matrix multiplications, non-linear functions like Softmax and layer normalization (LayerNorm) are increasingly dominating inference latency, especially for long sequences and high-dimensional inputs. To address this emerging bottleneck, we present a hardware accelerator that jointly approximates these non-linear functions using piecewise linear approximation for the exponential in Softmax and Newton–Raphson iteration for the square root in LayerNorm. The proposed unified architecture dynamically switches operation modes while reusing hardware resources. The proposed accelerator was implemented on a Xilinx VU37P FPGA and evaluated with BERT and GPT-2 models. Experimental results demonstrate speedups of up to 7.6× for Softmax and 2.0× for LayerNorm, while maintaining less than 1% accuracy degradation on classification tasks with conservative approximation settings. However, generation tasks showed greater sensitivity to approximation, underscoring the need for task-specific tuning.
On edge devices, convolutional neural network (CNN) inference is bottlenecked mainly by memory bandwidth, owing to the frequent memory accesses to feature maps and parameters. To address this challenge, we propose a memory-efficient hardware accelerator for depthwise separable convolution that minimizes off-chip memory traffic and parameter storage. The proposed architecture employs three key techniques: (1) a 64-bit run-length coding (RLC) packet compression that exploits feature-map sparsity after ReLU, (2) a mixed-precision scheme that represents feature maps and weights at different precisions, and (3) separated depthwise and pointwise convolution units. In particular, feature maps are transferred in RLC-compressed form, which reduces the amount of data exchanged with the host. The compressed data are decoded row by row, so the on-chip buffers hold only the rows required for computation rather than a complete feature map. In software simulation on ImageNet, the mixed-precision scheme reduced the parameter storage by 49.22% at a cost of 6.24 percentage points (pp) in Top-1 accuracy, and the RLC reduced the data by up to 56.07% in the deeper layers. Implemented on a Xilinx ZCU-104 FPGA, the proposed accelerator performs the depthwise separable convolution with a small number of logic resources and on-chip memory, confirming its feasibility for resource-constrained edge devices.