17 February 2025 in Neurocomputing
13 December 2025 in ACM Transactions on Architecture and Code Optimization
Optimizing deep learning inference, particularly reducing the execution latency of tensor computations at small batch sizes, is crucial for the successful and widespread adoption of deep neural network (DNN) models. However, current deep learning compilers and hand-tuned libraries often fail to achieve high hardware efficiency when executing small-batch workloads. The primary reason is the inherently sequential nature of reductions (e.g., along the hidden dimension in the flattened GEMM for LLM decoding), which is difficult to parallelize and therefore fails to fully utilize available hardware resources. In this article, we propose Swift, a novel search-based approach for efficiently generating high-performance programs for GPUs by maximizing hardware utilization. The key insight is that reduction parallelization can be incorporated into a unified representation alongside the existing tile structure, significantly expanding the search space for high-performance programs. Concretely, by enumerating all possible parallel mappings of loops, we first generate a large search space that contains high-performance programs. Then, to efficiently explore the extended search space, we employ subspace shifting exploration to identify promising regions, effectively prune large portions of the less-promising search space. We conduct experiments on three distinct GPU architectures using a diverse set of benchmarks representative of typical application scenarios. Experimental results demonstrate that Swift achieves an average speedup of 1.19× over the state-of-the-art compiler-based approaches. Moreover, compared with vendor-provided hand-tuned libraries, Swift achieves an average speedup of 2.40×.
1 November 2026 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Circuit pipelining is essential for improving the frequency and throughput of sequential circuits, yet it still largely relies on manual design at the behavioral level, which is effort-intensive, error-prone, and fails to exploit fine-grained gate-level opportunities. Existing automatic approaches analyze feedforward paths to insert registers but fall back to conservative stalling in the presence of feedback loops, sacrificing parallelism to preserve correctness. As a result, neither manual nor existing automatic methods simultaneously achieve fine-grained throughput optimization and correctness guarantees on arbitrary sequential circuits. We present an automatic pipelining framework that combines gate-level forwarding and customized scheduling. Our key insight is that gate-level analysis reveals shorter feedback loops, whose hazards can be resolved by forwarding paths spanning fewer pipeline stages, thus avoiding long multi-cycle dependencies that cause redundant stalls. Our method consists of two steps: the first focuses on fine-grained optimization beyond behavioral-level analysis, while the second ensures correctness for arbitrary circuits. Concretely, we (i) greedily assign pipeline stages at the gate level and create short forwarding paths for fine-grained pipeline optimization, and (ii) design a circuit-specific scheduler to ensure correctness. The scheduler analyzes state-conditioned reads/writes to predict future dependencies and apply only the necessary stalls, and is then synthesized into hardware using an improved Binary Speculative Diagram (BSD) algorithm with acceptable hardware overhead. Across diverse circuit datasets, our approach identifies 35.23% redundant pipeline stalls overlooked by behavioral-level manual designs and improves throughput by 9.46% for 3-stage and 23.07% for 4-stage pipelines over EDA-based automatic pipelining techniques, while guaranteeing functional correctness on arbitrary sequential circuits.
31 October 2025 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
Diffusion transformers (DiTs) have demonstrated remarkable success in text-to-video generation. However, the self-attention mechanism in DiTs imposes significant computational and memory burdens, particularly when handling long patch sequences like high-resolution or long-time videos. While sparse attention shows promise in reducing self-attention costs, existing approaches struggle to deliver performance gains due to the unique challenges in DiTs, i.e., varied sparse patterns across layers and timesteps, and the cumulative nature of inference errors over timesteps. In this article, we propose DASA, an algorithm-hardware co-design that effectively addresses these challenges of attention sparsification in DiTs. Specifically, leveraging the insight that the generation quality is primarily influenced by overall distribution drift rather than changes in specific values, we introduce a novel distribution-aware filtering (DAF) mechanism for sparsification. To further accelerate the process, we design a specialized filtering unit that enables fast candidate selection based on the proposed DAF mechanism. Experimental results show that DASA achieves $2.52\times $ speed up compared to A100 GPU, and up to $1.22\times $ speedup over state-of-the-art accelerators for self-attention computation.