Guanyu Feng

Guanyu Feng

Auto-generated profile, based on publicly available information

Guanyu Feng

ZHIPU AI, Beijing, China

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
28 March 2022
Verified
Conference Paper
Open access
ZMZixuan Ma
Zixuan Ma
JHJiaao He
Jiaao He
JQJiezhong Qiu
Jiezhong Qiu
HCHuanqi Cao
Huanqi Cao
YWYuanwei Wang
Yuanwei Wang
WCWenguang Chen
Wenguang Chen
Large-scale pretrained AI models have shown state-of-the-art accuracy in a series of important applications. As the size of pretrained AI models grows dramatically each year in an effort to achieve higher accuracy, training such models requires massive computing and memory capabilities, which accelerates the convergence of AI and HPC. However, there are still gaps in deploying AI applications on HPC systems, which need application and system co-design based on specific hardware features. To this end, this paper proposes BaGuaLu1, the first work targeting training brain scale models on an entire exascale supercomputer, the New Generation Sunway Supercomputer. By combining hardware-specific intra-node optimization and hybrid parallel strategies, BaGuaLu enables decent performance and scalability on unprecedentedly large models. The evaluation shows that BaGuaLu can train 14.5-trillion-parameter models with a performance of over 1 EFLOPS using mixed-precision and has the capability to train 174-trillion-parameter models, which rivals the number of synapses in a human brain.

27 April 2024
Verified
Conference Paper
Open access
ZSZhenbo Sun
Zhenbo Sun
HCHuanqi Cao
Huanqi Cao
YWYuanwei Wang
Yuanwei Wang
GFGuanyu Feng
Guanyu Feng
SCShengqi Chen
Shengqi Chen
WCWenguang Chen
Wenguang Chen
Large language models (LLMs) have demonstrated powerful capabilities, requiring huge memory with their increasing sizes and sequence lengths, thus demanding larger parallel systems. The broadly adopted pipeline parallelism introduces even heavier and unbalanced memory consumption. Recomputation is a widely employed technique to mitigate the problem but introduces extra computation overhead. This paper proposes AdaPipe, which aims to find the optimized recomputation and pipeline stage partitioning strategy. AdaPipe employs adaptive recomputation to maximize memory utilization and reduce the computation cost of each pipeline stage. A flexible stage partitioning algorithm is also adopted to balance the computation between different stages. We evaluate AdaPipe by training two representative models, GPT-3 (175B) and Llama 2 (70B), achieving up to 1.32× and 1.22× speedup on clusters with NVIDIA GPUs and Ascend NPUs respectively.

22 March 2023 in ACM Transactions on Storage
Verified
Journal Article
Open access
GFGuanyu Feng
Guanyu Feng
HCHuanqi Cao
Huanqi Cao
XZXiaowei Zhu
Xiaowei Zhu
BYBowen Yu
Bowen Yu
YWYuanwei Wang
Yuanwei Wang
WCWenguang Chen
Wenguang Chen
Out-of-core systems rely on high-performance cache sub-systems to reduce the number of I/O operations. Although the page cache in modern operating systems enables transparent access to memory and storage devices, it suffers from efficiency and scalability issues on cache misses, forcing out-of-core systems to design and implement their own cache components, which is a non-trivial task. This study proposes TriCache, a cache mechanism that enables in-memory programs to efficiently process out-of-core datasets without requiring any code rewrite. It provides a virtual memory interface on top of the conventional block interface to simultaneously achieve user transparency and sufficient out-of-core performance. A multi-level block cache design is proposed to address the challenge of per-access address translations required by a memory interface. It can exploit spatial and temporal localities in memory or storage accesses to render storage-to-memory address translation and page-level concurrency control adequately efficient for the virtual memory interface. Our evaluation shows that in-memory systems operating on top of TriCache can outperform Linux OS page cache by more than one order of magnitude, and can deliver performance comparable to or even better than that of corresponding counterparts designed specifically for out-of-core scenarios.

13 June 2023 in Proceedings of the ACM on Management of Data
Verified
Journal Article
Open access
ZPZhenxuan Pan
Zhenxuan Pan
TWTao Wu
Tao Wu
QZQingwen Zhao
Qingwen Zhao
QZQiang Zhou
Qiang Zhou
ZPZhiwei Peng
Zhiwei Peng
XZXiaowei Zhu
Xiaowei Zhu
GeaFlow is a distributed dataflow system optimized for streaming graph processing, and has been widely adopted at Ant Group, serving various scenarios ranging from risk control of financial activities to analytics on social networks and knowledge graphs. It is built on top of a base with full-fledged stateful stream processing capabilities, extended with a series of graph-aware optimizations to address the space explosion and programming complexity issues of conventional join-based approaches. We propose new state backends and streaming operators that facilitate processing on dynamic graph-structured datasets, reducing space consumed by states. We develop a hybrid domain-specific language that embeds Gremlin into SQL, supporting both table and graph abstractions over streaming data. In addition to streaming workloads, GeaFlow is also extensively used for some batch processing jobs. In the largest deployments to date, GeaFlow is able to process tens of millions of events per second and manage hundreds of terabytes of states.