1 January 2024 in Computer Modeling in Engineering & Sciences
Federated learning is an innovative machine learning technique that deals with centralized data storage issues while maintaining privacy and security. It involves constructing machine learning models using datasets spread across several data centers, including medical facilities, clinical research facilities, Internet of Things devices, and even mobile devices. The main goal of federated learning is to improve robust models that benefit from the collective knowledge of these disparate datasets without centralizing sensitive information, reducing the risk of data loss, privacy breaches, or data exposure. The application of federated learning in the healthcare industry holds significant promise due to the wealth of data generated from various sources, such as patient records, medical imaging, wearable devices, and clinical research surveys. This research conducts a systematic evaluation and highlights essential issues for the selection and implementation of federated learning approaches in healthcare. It evaluates the effectiveness of federated learning strategies in the field of healthcare. It offers a systematic analysis of federated learning in the healthcare domain, encompassing the evaluation metrics employed. In addition, this study highlights the increasing interest in federated learning applications in healthcare among scholars and provides foundations for further studies.
1 January 2025 in Computer Modeling in Engineering & Sciences
Federated Learning enables privacy-preserving training of Transformer-based language models, but remains vulnerable to backdoor attacks that compromise model reliability. This paper presents a comparative analysis of defense strategies against both classical and advanced backdoor attacks, evaluated across autoencoding and autoregressive models. Unlike prior studies, this work provides the first systematic comparison of perturbation-based, screening-based, and hybrid defenses in Transformer-based FL environments. Our results show that screening-based defenses consistently outperform perturbation-based ones, effectively neutralizing most attacks across architectures. However, this robustness comes with significant computational overhead, revealing a clear trade-off between security and efficiency. By explicitly identifying this trade-off, our study advances the understanding of defense strategies in federated learning and highlights the need for lightweight yet effective screening methods for trustworthy deployment in diverse application domains.
10 September 2025 in arXiv
Understanding how Transformer-based language models store and retrieve factual associations is critical for improving interpretability and enabling targeted model editing. Prior work, primarily on GPT-style models, has identified MLP modules in early layers as key contributors to factual recall. However, it remains unclear whether these findings generalize across different autoregressive architectures. To address this, we conduct a comprehensive evaluation of factual recall across several models -- including GPT, LLaMA, Qwen, and DeepSeek -- analyzing where and how factual information is encoded and accessed. Consequently, we find that Qwen-based models behave differently from previous patterns: attention modules in the earliest layers contribute more to factual recall than MLP modules. Our findings suggest that even within the autoregressive Transformer family, architectural variations can lead to fundamentally different mechanisms of factual recall.