23 September 2025 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Sparsity-aware techniques, traditionally applied to weight tensors, are increasingly utilized for activations in AI accelerators to enhance the energy efficiency and performance. While sparse accelerators promise significant performance gains, the impact of unpredictable activation sparsity on the performance variation and system uncertainty remains largely overlooked and so far has never been quantitatively analyzed, though it is a critical aspect for time-sensitive applications. Existing analytical models for sparse accelerators fail to capture the performance uncertainty due to their reliance on deterministic modeling of average performance, which assumes consistent hardware behavior and neglects variations arising from diverse sparsity patterns and levels. In this article, we propose SunPar, an open-source modeling framework for benchmarking and evaluating the actual performance variation of sparse AI accelerators under sparsity uncertainty. We categorize uncertainty in the sparsity patterns into two types: intratensor uncertainty (IntraU) and intertensor uncertainty (InterU). By constructing an analytical behavioral model, our framework can evaluate the performance variation in the form of probability density functions (PDFs), providing both the average performance and the performance uncertainty range with different confidence levels. Evaluation of SunPar across four sparse accelerators demonstrates its high accuracy, with a mismatch of <8% in the average performance estimation and <0.2% in performance variance estimation compared with the hardware results. In addition, two case studies provided for real-world applications based on image datasets (ImageNet) reveal a significant divergence between Pareto-optimal hardware choices with and without awareness of the performance uncertainty, underscoring the necessity of uncertainty-aware frameworks for designing sparse AI accelerators in time-sensitive applications.