8 August 2022 in IEEE Computer Architecture Letters
We propose to overcome the memory capacity limitation of GPUs with a Heterogeneous Memory Stack (HMS) that integrates Storage Class Memory (SCM) and DRAM in a 3D memory stack. By effectively utilizing the DRAM as a cache, the HMS addresses the latency, bandwidth, and energy challenges of the SCM. For workloads that mandate memory oversubscription, the HMS can capture a larger fraction of the memory footprint compared to an HBM, providing a significant performance uplift. To enable a performant DRAM cache architecture, we propose a DRAM cache bypass mechanism that avoids DRAM cache thrashing by 100,000 s of GPU threads. Furthermore, we propose a Tag Cache which uses part of L2 cache to hold tags of DRAM cachelines and reduce DRAM cache probe traffic, while enabling adjustment of L2 cache capacity used for Tag Cache. In addition, our DRAM cache organization enables fetching all DRAM cacheline tags in a row with a single column access, avoiding excessive DRAM cache probe traffic from Tag Cache misses. Our results show that the HMS outperforms the baseline HBM system under memory oversubscription by up to 9.8× (2.8× on average).
26 September 2022 in arXiv
Despite the evolution of Convolutional Neural Networks (CNNs), their performance is surprisingly dependent on the choice of hyperparameters. However, it remains challenging to efficiently explore large hyperparameter search space due to the long training times of modern CNNs. Multi-fidelity optimization enables the exploration of more hyperparameter configurations given budget by early termination of unpromising configurations. However, it often results in selecting a sub-optimal configuration as training with the high-performing configuration typically converges slowly in an early phase. In this paper, we propose Multi-fidelity Optimization with a Recurring Learning rate (MORL) which incorporates CNNs' optimization process into multi-fidelity optimization. MORL alleviates the problem of slow-starter and achieves a more precise low-fidelity approximation. Our comprehensive experiments on general image classification, transfer learning, and semi-supervised learning demonstrate the effectiveness of MORL over other multi-fidelity optimization methods such as Successive Halving Algorithm (SHA) and Hyperband. Furthermore, it achieves significant performance improvements over hand-tuned hyperparameter configuration within a practical budget.