Dong Qiang

Dong Qiang

Auto-generated profile, based on publicly available information

Dong Qiang

Ningxia University, Yinchuan, China

Research Focus & Metrics

Academic Career Length

1Year (2026-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
27 September 2026
Verified
Conference Paper
Open access
CMChanghao Ma
Changhao Ma
SWSuping Wu
Suping Wu
JDJiawei Dai
Jiawei Dai
JYJie Yang
Jie Yang
HXHao Xu
Hao Xu
DQDong Qiang
Dong Qiang
Activation recomputation (gradient checkpointing) reduces the memory footprint of intermediate activations from O(n) to \(O(\sqrt {n})\) by selectively discarding and recomputing them during the backward pass. However, a fundamental limitation persists: all existing implementations follow a synchronous blocking paradigm, forcing recomputation to fully occupy the critical path of backpropagation and typically introducing 30%–40% additional training overhead. This paper proposes Async-Checkpoint, an asynchronous activation recomputation algorithm. Our core insight is that the serial dependency between the recomputation and gradient computation of adjacent blocks is not required by data flow. Based on this, we redesign the execution logic—replacing the serial “recompute-then-gradient” loop with an asynchronous pipeline that overlaps the two tasks in separate CUDA streams, thereby removing recomputation from the critical path. Experiments on BERT, ResNet, U-Net, and a recurrent MVS model demonstrate generality across linear chains, DAGs with skip connections, and dynamic recurrent structures. Async-Checkpoint consistently reduces iteration time compared to the PyTorch synchronous implementation while maintaining memory consumption on par with synchronous checkpointing.