This survey systematically reviews Low-Rank Adaptation (LoRA) methods and their variants for Large Language Models (LLMs), emphasizing their efficiency in adapting models without compromising performance. We analyze recent advancements, technical foundations, and practical applications of these methods. By comparing LoRA with full fine-tuning, we highlight that although LoRA often underperforms on typical benchmark tasks, it effectively mitigates catastrophic forgetting and preserves generalization to tasks beyond the target domain. Additionally, this survey discusses advanced approaches, including robust adaptation, sparse fine-tuning, and hybrid techniques, along with in-depth examinations of spectral properties, generalization behaviors, and intruder dimensions. We identify significant progress, clarify current methodological challenges, and propose future research directions, providing a concrete and actionable guide for enhancing parameter-efficient fine-tuning of large-scale language models.

We propose a novel type of Artificial Immune System (AIS): Symbiotic Artificial Immune Systems (SAIS), drawing inspiration from symbiotic relationships in biology. SAIS parallels the three key stages (i.e., mutualism, commensalism and parasitism) of population updating from the Symbiotic Organisms Search (SOS) algorithm. This parallel approach effectively addresses the challenges of large population size and enhances population diversity in AIS, which traditional AIS and SOS struggle to resolve efficiently. We conducted a series of experiments, which demonstrated that our SAIS achieved comparable performance to the state-of-the-art approach SOS and outperformed other popular AIS approaches and evolutionary algorithms across 26 benchmark problems. Furthermore, we investigated the problem of parameter selection and found that SAIS performs better in handling a relatively large population size while requiring fewer generations. Finally, we believe SAIS, as a novel bio-inspired and immune-inspired algorithm, paves the way for innovation in bio-inspired computing with the symbiotic paradigm.

Zhongyu Yang, Junhao Song, Siyang Song, Wei Pang, Yingfang Yuan. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing. 2025.

This paper provides a comprehensive and structured overview of learning paradigms designed to operate under limited supervision, including few-shot, zero-shot, and weak-shot learning. Examines foundational concepts such as episodic training, meta-learning frameworks, representation learning, and recent advances in weakly supervised and unsupervised settings. Core methodologies, including transfer learning, multitask learning, and attention-based architectures, are discussed in detail, along with theoretical perspectives that analyze the conditions under which these approaches succeed. The paper also evaluates current benchmarks, datasets, and evaluation protocols such as LibFew-Shot, emphasizing the need for standardization and reproducibility. Applications in computer vision, natural language processing, and medical imaging demonstrate the real-world impact and versatility of these techniques. By synthesizing existing work and highlighting key challenges and future directions, this article serves as both a foundational reference and a roadmap to advance the field of low-data learning.

This paper examines emerging solutions for efficient medical Artificial Intelligence (AI) computing, exploring the convergence of AI with innovative technologies like silicon photonics, edge AI, distributed machine learning, and parallelism techniques to revolutionize healthcare. Covering applications in natural language processing, computer vision, and graph-structured data, the paper emphasizes the need for domain-specific foundation models, rigorous validation, and scalable computing solutions. It highlights human-AI collaboration in scientific discovery and decision-making, while stressing the importance of considering domain constraints when selecting hardware for medical AI deployment. This survey provides valuable insights for advancing innovative, efficient, and trustworthy AI healthcare solutions, contributing to the broader goal of improving human health and wellbeing.