Federated learning (FL), which provides a collaborative training scheme for distributed data sources with privacy concerns, has become a burgeoning and attractive research area. Most existing FL studies focus on taking unimodal data, such as image and text, as the model input and resolving the heterogeneity challenge, i.e., the challenge of non-identical distribution (non-IID) caused by a data distribution imbalance related to data labels and data amount. In real-world applications, data are usually described by multiple modalities. However, to the best of our knowledge, only a handful of studies have been conducted to improve system performance utilizing multimodal data. In this survey paper, we identify the significance of this emerging research topic of multimodal federated learning (MFL) and present a literature review on the state-of-art MFL methods. Furthermore, we categorize multimodal federated learning into congruent and incongruent multimodal federated learning based on whether all clients possess the same modal combinations. We investigate the feasible application tasks and related benchmarks for MFL. Lastly, we summarize the promising directions and fundamental challenges in this field for future research.

Predicting patients' risk of developing certain diseases is an important research topic in healthcare. Accurately identifying and ranking the similarity among patients based on their historical records is a key step in personalized healthcare. The electric health records (EHRs), which are irregularly sampled and have varied patient visit lengths, cannot be directly used to measure patient similarity due to the lack of an appropriate representation. Moreover, there needs an effective approach to measure patient similarity on EHRs. In this paper, we propose two novel deep similarity learning frameworks which simultaneously learn patient representations and measure pairwise similarity. We use a convolutional neural network (CNN) to capture local important information in EHRs and then feed the learned representation into triplet loss or softmax cross entropy loss. After training, we can obtain pairwise distances and similarity scores. Utilizing the similarity information, we then perform disease predictions and patient clustering. Experimental results show that CNN can better represent the longitudinal EHR sequences, and our proposed frameworks outperform state-of-the-art distance metric learning methods.

Xiaochen Wang, Junyu Luo, Jiaqi Wang, Yuan Zhong, Xiaokun Zhang, Yaqing Wang, Parminder Bhatia, Cao Xiao, Fenglong Ma. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2024.

The discovery of Wireless Power Transfer (WPT) technologies makes charging more convenient and reliable. Among all the existing WPT technologies, directional WPT is more efficient and has been successfully applied to supply energy for wireless rechargeable sensor networks (WRSNs). However, the state-of-the-art methods ignore the anisotropic energy receiving property of rechargeable sensors, resulting in energy wastage. In order to address this issue, in this paper, we point out that the received energy of a sensor is not only relative to the distance, but also relative to the angle between the sensor and the charger's orientation in directional WPT. Towards this end, we derive a pragmatic energy transfer model verified by experiments. In particular, we focus on a Minimal chArging Delay (MAD) problem to reduce charging delays. To obtain the optimal solution, we formulate the problem as a linear programming problem. Moreover, we introduce a method of charging power discretization, which significantly reduces the search space and bounds the performance gap to the optimal one with a 1/1-ϵ ² approximation ratio. Besides, a merging method is introduced for a more practical application scenario. Finally, we demonstrate that our methods outperform the Set Cover baseline method by an average of 34.2% through simulations and experiments.