Deep learning-driven deep recommendation systems have achieved remarkable success in recent years. However, the deployment of deep recommendation models on resource-constrained equipment and systems (e.g., mobile devices and embedded systems) is a significant challenge. To overcome this challenge, knowledge distillation has been introduced into the design of deep recommendation algorithms as a typical model compression and acceleration technique, and has gradually attracted the attention of both academia and industry. In this survey, the main efforts are: (1) to discuss the basic concepts, necessity and operation mechanism issues related to knowledge distillation recommendation systems; (2) to summarize and categorize existing representative knowledge distillation recommendation methods according to the viewpoint of knowledge distillation, and then analyze and elaborate their representative knowledge distillation recommendation methods in depth; (3) to introduce the analysis and discussion done by industry on how knowledge distillation can be applied to industrial recommendation systems, and summarize the ideas of applying knowledge distillation in various links of industrial recommendation systems; and (4) to present several possible future research directions for knowledge distillation recommendation systems, and summarize the results of the survey. This survey can guide researchers and practitioners to prepare and encourage further efforts to advance the field.
Food recommendation systems play a pivotal role in shaping dietary salubrity and fostering sustainable lifestyles by recommending recipes and foodstuffs that align with user preferences. Metadata information of a recipe, encompassing multi-modal descriptions, constituent ingredients, and health-related attributes, can furnish a more holistic perspective on the recipe's profile, thereby augmenting recommendation performance. However, existing state-of-the-art methods often overlook the inherent interdependencies between modalities, ingredients, and health factors, leaving the health information pertaining to recipe characteristics underexploited. Notably, our preliminary investigation on two datasets unveiled that the semantic divergence between health-related knowledge and collaborative filtering signals is more pronounced in comparison to other metadata information, thereby potentially impeding the efficacy of food recommendation systems. To address these limitations, we propose HealthRec, a novel multi-modal food recommendation framework with health-aware knowledge distillation. HealthRec employs a global graph representation learning module to capture high-order dependencies across diverse food-related relations, enriching the representations. Subsequently, a co-attention network is leveraged to capture local, recipe-level knowledge transfer between modality-related and ingredient-related embeddings. Additionally, we exploit external supervision signals derived from WHO recommendations, utilizing knowledge distillation during the training phase to transfer local health-aware knowledge into global collaborative embeddings. Extensive experimentation on real-world datasets demonstrates HealthRec's superiority compared to current state-of-the-art recommendation baselines, highlighting its effectiveness in modeling health-aware food recommendations.
16 April 2025 in IEEE Journal of Biomedical and Health Informatics
The proliferation of healthcare data sources, including diverse imaging modalities and biochemical measurements, has created unprecedented opportunities for comprehensive disease prediction. Multi-modal clinical data, encompassing medical imaging reports, biochemical assays, and longitudinal clinical records, provides a rich foundation for developing sophisticated diagnostic models. Graph Neural Networks (GNNs) have emerged as a leading methodological framework, distinguished by their capacity to model complex inter-patient relationships and capture community structures within patient data. Despite their promise, current GNN-based approaches exhibit limitations in handling noisy, low-quality data and often impose overly restrictive graph smoothness constraints. These limitations can obscure patient-specific variations and compromise model robustness. To overcome these challenges, we propose HierSSL (Hierarchical Self-Supervised Learning), a novel multi-modal disease prediction framework that enhances representational learning through dual-scale self-supervision mechanisms operating at both local and global levels. HierSSL's architecture specifically addresses two critical aspects: 1) the capture of local inter-modality dependencies and global community patterns, and 2) the optimization of multi-modal feature integration through an innovative combination of feature consistency constraints and graph contrastive learning. Empirical evaluation across two distinct disease prediction datasets demonstrates that HierSSL achieves statistically significant performance improvements compared to state-of-the-art methods, highlighting its efficacy in robust multi-modal data integration for disease prediction tasks.
Abnormal speech prosody has been widely reported in individuals with autism. Many studies on children and adults with autism spectrum disorder speaking a non-tonal language showed deficits in using prosodic cues to mark focus. However, focus marking by autistic children speaking a tonal language is rarely examined. Cantonese-speaking children may face additional difficulties because tonal languages require them to use prosodic cues to achieve multiple functions simultaneously such as lexical contrasting and focus marking. This study bridges this research gap by acoustically evaluating the use of Cantonese speech prosody to mark information structure by Cantonese-speaking children with and without autism spectrum disorder. We designed speech production tasks to elicit natural broad and narrow focus production among these children in sentences with different tone combinations. Acoustic correlates of prosodic focus marking like f0, duration and intensity of each syllable were analyzed to examine the effect of participant group, focus condition and lexical tones. Our results showed differences in focus marking patterns between Cantonese-speaking children with and without autism spectrum disorder. The autistic children not only showed insufficient on-focus expansion in terms of f0 range and duration when marking focus, but also produced less distinctive tone shapes in general. There was no evidence that the prosodic complexity (i.e. sentences with single tones or combinations of tones) significantly affected focus marking in these autistic children and their typically-developing (TD) peers.
3 February 2025 in Expert Systems with Applications