Jiadi Chen

Jiadi Chen

Auto-generated profile, based on publicly available information

Jiadi Chen

Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China

Research Focus & Metrics

Academic Career Length

2Years (2025-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
14 February 2025 in IEEE Access
Verified
Journal Article
Open access
NJNi Jia
Ni Jia
TLTong Liu
Tong Liu
JCJiadi Chen
Jiadi Chen
YZYing Zhang
Ying Zhang
SHSong Han
Song Han
The advent of Large Language Models (LLMs) has revolutionized Natural Language Processing (NLP), offering unprecedented capabilities for understanding and generating human-like texts. However, the effectiveness of these models in specific domains often requires fine-tuning, which can be computationally expensive and increases the risk of overfitting. To address these challenges, we introduce Task-Agnostic Adaptive Activation Scaling Network (TAAS-Net), a novel multi-task learning model architecture designed to adapt pre-trained LLMs to new tasks with minimal computational and storage overhead. TAAS-Net leverages an adaptive activation scaling network to enhance parameter efficiency and prevent overfitting, thereby improving model generalization. It incorporates a task-agnostic adaptive activation scaling layer that introduces task-specific, learnable activation scaling vectors, enhancing the model’s adaptability across various tasks. In addition, Low-Rank Adaptation within the attention modules facilitates the transfer of knowledge across different tasks. TAAS-Net’s integration of multi-task learning and parameter-efficient fine-tuning schemes enables efficient fine-tuning in multi-task scenarios, with a reduced parameter count compared to existing methods such as MoE-LoRA and X-LoRA. Our model also handles mixed-task batches during inference, thereby enhancing versatility and operational efficiency. Through a series of experiments on benchmark datasets, including the General Language Understanding Evaluation (GLUE), IMDB, and Agnews, TAAS-Net demonstrated superior performance over existing baselines, underscoring its potential for applications requiring rapid deployment and high accuracy. The architectural design of TAAS-Net streamlines the model adaptation process and reduces resource demands, offering a scalable and resource-efficient solution for adapting LLMs to new tasks.