16 June 2023 in ACM Transactions on Asian and Low-Resource Language Information Processing
The Chinese radiology report summarization is a crucial component in smart healthcare that employs language models to summarize key findings in radiology reports and communicate these findings to physicians. However, most language models for radiology report summarization utilize a softmax transformation in their output layer, leading to dense alignments and strictly positive output probabilities. This density is inefficient, reducing model interpretability and giving probability mass to many unrealistic outputs. To tackle this issue, we propose a novel approach named nucleusmax. Nucleusmax is able to mitigate dense outputs and improve model interpretability by truncating the unreliable tail of the probability distribution. In addition, we incorporate nucleusmax with a copy mechanism, a useful technique to avoid professional errors in the generated diagnostic opinions. To further promote the research of radiology report summarization, we also have created a Chinese radiology report summarization dataset, which is freely available. Experimental results showed via both automatic and human evaluation that the proposed approach substantially improves the sparsity and overall quality of outputs over competitive softmax models, producing radiology summaries that approach the quality of those authored by physicians. In general, our work demonstrates the feasibility and prospect of the language model to the domain of radiology and smart healthcare.
8 July 2023 in Knowledge-Based Systems
19 March 2024 in Pattern Recognition Letters
8 January 2025 in IEEE Transactions on Information Forensics and Security
Federated learning faces challenges associated with privacy breaches, client communication efficiency, stragglers’ effect, and heterogeneity. To address these challenges, this paper reformulates the optimal client selection problem as a sparse optimization task, proposes a secure and efficient optimal client selection method for federated learning, named secure orthogonal matching pursuit federated learning (SecOMPFL). Therein, we first introduce a method to identify correlations in the local model parameters of participating clients, addressing the issue of duplicated client contributions highlighted in recent literature. Next, we establish a secure variant of the OMP algorithm in compressed sensing using secure multiparty computation and propose a novel secure aggregation protocol. This protocol enhances the global model’s convergence rate through sparse optimization techniques while maintaining privacy and security. It relies entirely on the local model parameters as inputs, minimizing client communication requirements. We also devise a client sampling strategy without requiring additional communication, resolving the bottleneck encountered by the optimal client selection policy. Finally, we introduce a strict yet inclusive straggler penalty strategy to minimize the impact of stragglers. Theoretical analysis confirms the security and convergence of SecOMPFL, highlighting its resilience to stragglers’ effect and systematic/statistical heterogeneity with high client communication efficiency. Numerical experiments were conducted to compare the convergence rate and client communication efficiency of SecOMPFL with those of FedAvg, FOLB, and BN2. These experiments used natural and synthetic with statistical heterogeneity datasets, considering varying numbers of clients and client sampling scales. The results demonstrate that SecOMPFL achieves a competitive convergence rate, with communication overhead 39.96% lower than that of FOLB and 28.44% lower than that of BN2. Furthermore, SecOMPFL shows good resilience to statistical heterogeneity.