Gang Lu

Gang Lu

Auto-generated profile, based on publicly available information

Gang Lu

China Telecom Research Institute, Guangzhou, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
18 November 2024 in IEEE Access
Verified
Journal Article
Open access
RLRui Li
Rui Li
DFDeji Fu
Deji Fu
CSChunyu Shi
Chunyu Shi
ZHZhilan Huang
Zhilan Huang
GLGang Lu
Gang Lu
ChatGPT was released in late November 2022, making a significant impact globally. Following this release, numerous domestic and international open-source projects for large model training emerged, including Alpaca, BOOLM, LLaMA, ChatGLM, DeepSpeedChat, and ColossalChat. Both academia and industry have a growing need to train large models for optimizing downstream tasks. Research has demonstrated that instruction tuning using high-quality training data can enable models with over 10 billion parameters to exhibit emergent abilities, particularly in complex reasoning tasks. Training a model with tens of billions of parameters is often necessary to improve business metrics using large model technology. However, most research teams lack the extensive computing resources required for such large-scale models, often possessing only a few V100 (32G) GPUs and occasionally a few A100s. Consequently, training or inferring a large model of this magnitude is impractical under limited computational resources. Therefore, adopting optimization strategies during the training and inference stages is essential to address these challenges. This survey summarizes a series of optimization techniques for large models during the training and inference phases, enabling the training of large models even with constrained computational resources.

26 March 2025 in IEEE Internet of Things Journal
Verified
Journal Article
Closed access
HMHuirong Ma
Huirong Ma
ZHZhilan Huang
Zhilan Huang
GLGang Lu
Gang Lu
DFDeji Fu
Deji Fu
CSChunyu Shi
Chunyu Shi
Machine-learning (ML) inference services are vital for artificial intelligence (AI) applications on edge devices. However, there are challenges to balance between maintaining high inference accuracy and minimizing carbon emissions. To foster green AI services, we introduce a novel framework that taps into green energy sources, empowering Internet of Things (IoT) devices to execute inference tasks at the network’s edge. In the context of multitask concurrent scheduling, our methodology seeks to simultaneously optimize both the inference accuracy and carbon footprint of all inference tasks. To support this endeavor, we propose a unique incentive constraint that ensures equitable allocation of computing resources for IoT devices equipped with energy harvesting modules, thereby promoting efficient collaboration. Furthermore, recognizing the system’s dynamic nature and fluctuating availability of green energy, we introduce the carbon-aware online inference task offloading algorithm (OTOA). This algorithm, anchored in the Lyapunov optimization method, leverages a graph-based offloading strategy, amalgamating elements from the two-sided matching method, to derive an approximate optimal solution using real-time data. Our meticulous theoretical evaluations and in-depth experiments with real-world datasets affirm OTOA’s superiority over existing approaches.

30 May 2024 in IEEE Transactions on Emerging Topics in Computing
Verified
Journal Article
Closed access
RLRongkai Liu
Rongkai Liu
YWYuting Wu
Yuting Wu
KZKongyange Zhao
Kongyange Zhao
ZZZhi Zhou
Zhi Zhou
XGXiang Gao
Xiang Gao
GLGang Lu
Gang Lu
Driven by the accelerated convergence of artificial intelligence (AI) and the Internet of Things (IoT), the recent years have witnessed the booming of Artificial Intelligence of Things (AIoT). Edge clouds place computing and service capabilities at the network edges to reduce network transmission overhead, which has been widely recognized as the critical infrastructure for AIoT applications. Meanwhile, to accelerate computationintensive edge cloud AI operations, specialized AI accelerators such as GPU, NPU, and TPU have been increasingly integrated into edge clouds. For such emerging XPU edge clouds, utilizing costly XPUs more efficiently has become a significant challenge. In this paper, we present an online optimization framework for joint resource provisioning and batch scheduling for more costefficient AIoT inference serving in an XPU edge cloud. The essential optimization process for the online framework is to first adaptively batch inference tasks to increase the system throughput without compromising the service level agreement (SLA). Next, heterogeneous XPU resources are provisioned for the batches. Finally, the resource instance is consolidated to a minimum of physical servers. Via extensive trace-driven simulations, we verify the performance of the presented online optimization framework.