13 March 2025 in ACM Transactions on Management Information Systems
The surge in interest and application of large language models (LLMs) has sparked a drive to fine-tune these models to suit specific applications, such as finance and medical science. However, concerns regarding data privacy have emerged, especially when multiple stakeholders aim to collaboratively enhance LLMs using sensitive data. In this scenario, federated learning becomes a natural choice, allowing decentralized fine-tuning without exposing raw data to central servers. Motivated by this, we investigate how data privacy can be ensured in LLM fine-tuning through practical federated learning approaches, enabling secure contributions from multiple parties to enhance LLMs. Yet, challenges arise: (1) despite avoiding raw data exposure, there is a risk of inferring sensitive information from model outputs, and (2) federated learning for LLMs incurs notable communication overhead. To address these challenges, this article introduces DP-LoRA, a novel federated learning algorithm tailored for LLMs. DP-LoRA preserves data privacy by employing a Gaussian mechanism that adds noise in weight updates, maintaining individual data privacy while facilitating collaborative model training. Moreover, DP-LoRA optimizes communication efficiency via low-rank adaptation, minimizing the transmission of updated weights during distributed training. The experimental results across medical, financial, and general datasets using various LLMs demonstrate that DP-LoRA effectively ensures strict privacy constraints while minimizing communication overhead.
31 July 2024 in IEEE Transactions on Instrumentation and Measurement
Heart rate measurement based on radar is an effective method for non-contact monitoring of human vital signs. However, the heart rate signal obtained from the radar is subject to noise interference due to the imperfect circuit characteristics. To tackle this challenge in heart rate detection by micro-Doppler radar, this paper employs a narrow-beam radar operating in the 120 GHz band to capture the heartbeat signal and adopts an intelligent singular value decomposition algorithm to effectively eliminate the noise in the heartbeat signal. This approach can adaptively select the number of large singular values corresponding to the best noise reduction effect under different signal-to-noise ratios. In addition, an improved automatic multi-scale peak detection algorithm is proposed for precise peak detection in the heartbeat signal. Experimental results demonstrate that the proposed algorithm significantly reduces omission or misjudgment during the peak detection process, leading to a high accuracy during short-term heart rate detection. Compared with the pulse wave measurement results, the average accuracy of the heart rate is between 96.53% and 98.18%.