Advances in deep learning have revolutionized medical image segmentation, facilitating the precise delineation of complex anatomical structures. The scarcity of annotated training samples remains a significant bottleneck. To tackle the data limitation, federated learning (FL) offers the promise of pooling data from multiple healthcare institutions. However, as models grow larger, the increase in communication costs restricts FL to fewer nodes, which constrains the volume of data. This situation necessitates the simultaneous achievement of model lightweighting. To address this problem, this study proposes FKD-Med, a novel framework that integrates FL for privacy-sensitive data amalgamation across multiple healthcare institutions, and uses knowledge distillation (KD) to enhance communication efficiency. The "Med" in FKD-Med refers to medical application computational problems. Our principal contributions encompass the design of an open-source framework that seamlessly blends FL and KD, rendering it applicable to a broad spectrum of medical informatics tasks. Our approach substantially augments the computational data volume, thereby boosting both communication efficiency and training throughput. Tested on two datasets of medical image segmentation using TransUNet and ResUNet as teacher models, FKD-Med achieves data privacy, lowers communication costs, and increases accuracy. The parameters of the student models were reduced to 1/127 and 1/1027 of those in the teacher models. Additionally, the models subjected to KD exhibited accuracy improvements of 0.25%, 0.43%, 1.35%, and 1.46% respectively, given the same parameter volume. This positions FKD-Med not only as a pivotal tool for multi-institutional medical research but also as a versatile platform adaptable to a wide array of real-world medical engineering applications. The code is publicly available at https://github.com/SUN-1024/FKD-Med.
A great deal of information related to rice cultivation has been published on the web. Conventionally, this information is studied by end-users to identify pests, and to prevent production losses from rice diseases. Despite its benefits, such information has not yet been encoded in a machine-processable form. This research closes the gap by modeling the knowledge-bases using ontologies and semantic technologies. Our modeled ontologies are externalized from existing reliable sources only, and offer axioms that describe abnormal appearances in rice diseases (and insects) and the corresponding controls. In addition, we developed an expert system called RiceMan, based on our ontologies, to support technical and non-technical users for diagnosing problems from observed abnormalities. We also introduce a composition procedure that aggregates users’ observation data with others for realizing spreadable diseases. This procedure, together with ontology reasoning, lies at the heart of our methodology. Finally, we evaluate our methodology practically with four groups of stakeholders in Thailand: senior agronomists, junior agronomists, agricultural students, and ontology specialists. Both ontologies and RiceMan are evaluated to verify their correctness, usefulness, and usability in various aspects. Our experimental results show that ontology reasoning is a promising approach for this domain problem.
Data-driven and knowledge-driven methods are two mainstream techniques in the pursuit of developing artificial intelligence systems. While data-driven methods seek to develop a decision model from observations in the real world, they are difficult to provide an explanation for the results in human terms. On the other hand, knowledge-driven methods that employ symbolic reasoning based on formal semantics of a knowledge-base are thus more interpretable and explainable, while lacking an ability to deal with incomplete modeling of the structured knowledge-bases. This work aims to tackle these issues on ontology similarity by proposing a general framework that combines the strengths of both approaches for measuring semantic similarity of concepts in a description logic (DL) ontology. More specifically, a neuro-symbolic integrated framework is defined to exploit the pre-trained word embeddings with semantic definitions in an ontology to yield an explainable degree of concept similarity. To demonstrate its applicability, we develop a concrete similarity measure $ {\textsf {sim}}_\epsilon $ conforming to the proposed framework and also introduce an efficient algorithm that can extract an explanation for why such a degree is indicated. The correctness is shown by analyzing theoretical properties that it guarantees to preserve and also by performing an empirical evaluation with a medical ontology SNOMED CT and a medical pre-trained embedding BioWordVec. The results show that our proposed method remains both interpretability and explainability while achieving comparable performance, relative to the state-of-the-art approaches in the data and knowledge-driven methods.
Sign Language Recognition (SLR) plays a critical role in bridging communication gaps between the deaf and hearing communities. This research introduces GSR-Fusion, a deep multimodal fusion architecture that combines RGB-based, skeleton-based, and graph-based modalities to enhance gesture recognition. Unlike traditional unimodal models, GSR-Fusion utilizes gesture initiation and termination detection, along with a cross-modality fusion approach using a merge (network) technique, enabling it to capture both spatial-temporal and relational features from multiple data sources. The model incorporates ViViT for RGB feature extraction, Transformers for sequential pose modeling, and A3T-GCN for joint graph representation, which together form a comprehensive understanding of gestures. The study investigates five key experimental setups, covering single-hand static and dynamic gestures (one, two, and three strokes) as well as two-hand static and dynamic gestures from the Thai Finger Spelling dataset. Additionally, we compare our architecture with existing models on global datasets, including WLASL and MS-ASL, to evaluate its performance. The results show that GSR-Fusion outperforms state-of-the-art models on multiple datasets. On WLASL, it achieves 83.45% accuracy for 100 classes and 75.23% for 300 classes, surpassing models like SignBERT and Fusion3. Similarly, on MS-ASL, it attains 84.31% for 100 classes and 80.57% for 200 classes, outperforming both RGB-based and skeleton-based models. These results highlight the effectiveness of GSR-Fusion in recognizing complex gestures, demonstrating its ability to generalize across different sign languages and datasets. The research emphasizes the importance of multimodal fusion in advancing sign language recognition for real-world applications.