The research of multimodal emotion recognition based on facial expressions, speech, and body gestures is crucial for oncoming intelligent human–computer interfaces. However, it is a very difficult task and has seldom been researched in this combination in the past years. Based on the GEMEP and Polish databases, this contribution focuses on trimodal emotion recognition from facial expressions, speech, and body gestures, including feature extraction, feature fusion, and multimodal classification of the three modalities. In particular, for feature fusion, two novel algorithms including supervised least squares multiset kernel canonical correlation analysis (SLSMKCCA) and sparse supervised least squares multiset kernel canonical correlation analysis (SSLSMKCCA) are presented, respectively, to carry out efficient facial expression, speech, and body gesture feature fusion. Different from the traditional multiset kernel canonical correlation analysis (MKCCA) algorithms, our SLSKMCCA algorithm is a supervised version and is based on the least squares form. The SSLSKMCCA algorithm is implemented by the combination of SLSMKCCA and a sparse item (L1 Norm). Moreover, two effective solving algorithms for SLSMKCCA and SSLSMKCCA are presented in addition, which use the alternated least squares and augmented Lagrangian multiplier methods, respectively. The extensive experimental results on the popular public GEMEP and Polish databases show that the recognition rate of multimodal emotion recognition is superior to bimodal and monomodal emotion recognition on average, and our presented SLSMKCCA and SSLSMKCCA fusion methods both obtain very high recognition rates, especially for the SSLSMKCCA fusion method.

The Graph Convolutional Neural Networks (GCN) method has shown excellent performance in the field of deep learning, and using graphs to represent speech data is a computationally efficient and scalable approach. In order to enhance the adequacy of graph neural networks in extracting speech emotional features, this paper proposes a Temporal-Spatial Learnable Graph Convolutional Neural Network (TLGCNN) for speech emotion recognition. TLGCNN firstly utilizes the Open-SMILE toolkit to extract frame-level speech emotion features. Then, a bidirectional long short-term memory (Bi LSTM) network is used to process the long-term dependencies of speech features which can further extract deep frame-level emotion features. The extracted frame-level emotion features are then input into subsequent network through two pathways. Finally, one pathway constructs the extracted frame-level deep emotion feature vectors into a graph structure applying an adaptive adjacency matrix to catch latent spatial connections, while the other pathway concatenates emotion feature vectors with graph-level embedding obtained from learnable graph convolutional neural network for prediction and classification. Through these two pathways, TLGCNN can simultaneously obtain temporal speech emotional information through Bi-LSTM and spatial speech emotional information through Learnable Graph Convolutional Neural (LGCN) network. Experimental results demonstrate that this method achieves weighted accuracy of 66.82% and 58.35% on the IEMOCAP and MSP-IMPROV databases, respectively.

In order to obtain more fine-grained information from multiple sub-feature spaces for domain adaptation, this paper proposes a novel multi-representation joint dynamic domain adaptation network (MJDDAN) and applies it to achieve cross-database facial expression recognition. The MJDDAN uses a hybrid structure to extract multi-representation features and maps the original facial expression features into multiple sub-feature spaces, aligning the expression features of the source domain and target domain in multiple sub-feature spaces from different angles to extract features more comprehensively. Moreover, the MJDDAN proposes the Joint Dynamic Maximum Mean Difference (JD-MMD) model to reduce the difference in feature distribution between different subdomains by simultaneously minimizing the maximum mean difference and local maximum mean difference in each substructure. Three databases, including eNTERFACE, FABO, and RAVDESS, are used to design a large number of cross-database transfer learning facial expression recognition experiments. The accuracy of emotion recognition experiments with eNTERFACE, FABO, and RAVDESS as target domains reach 53.64%, 43.66%, and 35.87%, respectively. Compared to the best comparison method chosen in this article, the accuracy rates were improved by 1.79%, 0.85%, and 1.02%, respectively.

Edge drones continuously collect sensitive information such as telemetry data during missions, making it difficult to apply centralized model training directly due to privacy protection, security compliance, and regulatory constraints. Although federated learning (FL) can avoid sharing raw data, existing federated learning schemes based solely on homomorphic encryption (HE) still face security risks in drone scenarios, such as gradient inversion, member inference, and malicious update injection. To address this, we propose a secure and verifiable edge federated learning framework for parameter-efficient model adaptation in drone scenarios. The framework introduces homomorphic encryption for model updates on the device side to protect the privacy of updates before transmission and aggregation. Simultaneously, on the server side, decryption, aggregation, and verification are performed through a remotely authenticated Trusted Execution Environment (TEE), thereby limiting the server's access to plaintext updates and reducing the feasibility of gradient inversion and member inference attacks at the system level. Furthermore, an aggregation signature mechanism is introduced to batch verify the identity and update integrity of participating nodes, effectively preventing malicious or tampered updates from participating in aggregation, thus overcoming the shortcomings of existing HE-FL schemes in terms of poisoning resistance and verifiability. Experimental results show that, while ensuring safety and verifiability, the proposed method improves model accuracy by 3% compared to the comparative scheme, while maintaining better performance in terms of computation and communication overhead, thus verifying the practicality and deployability of the framework in resource-constrained UAV edge environments.

To address the security challenges of untrusted servers and the difficulty in verifying malicious node identities in UAV swarm federated learning when relying solely on homomorphic encryption (HE), this paper proposes a verifiable HE-based federated learning (VHEFL) framework that integrates trusted execution environments (TEEs) with aggregate signatures. The method proceeds through a series of well-defined steps. Initially, each UAV computes model gradients locally, encrypts them using additive HE, and attaches a lightweight identity-based digital signature. The encrypted model gradients and signatures are uploaded to the server. Subsequently, the ground server performs gradient aggregation, including both the encrypted gradients and signatures, in the regular execution environment (REE). After performing aggregation in the REE, the aggregated ciphertext and all the signatures are sent to the TEE. Inside the TEE, an efficient aggregate verification algorithm is executed to batch-verify the authenticity and integrity of all signatures simultaneously. This enables the TEE to securely verify the aggregated result and return the aggregated model parameters to the UAVs, where the encrypted model parameters are decrypted locally to obtain the updated parameters. By combining low-overhead aggregate signature verification with hardware isolation, VHEFL provides a high-performance and verifiable security solution that effectively addresses the challenges of data privacy, malicious update prevention, and computational overhead in UAV swarm federated learning.

Background: Tacit knowledge plays a crucial role in clinical decision-making and medical innovation, particularly through experience-based and practice-oriented expertise. However, existing research has not yet provided a sufficiently integrated framework to explain how such knowledge is structured and transformed within medical practice. Methods: Grounded in embodied cognition theory, this study constructs a medical experts' tacit knowledge model encompassing four key elements of expert agent, context, thinking, and action. Building upon the layered perspective of the onion model, the study organizes tacit knowledge across three levels and explains its dynamic and bidirectional transformation. Results: The resulting framework integrates embodied experience, cognitive processes, and clinical practice into a coherent system. A case analysis of the acupuncture expert Wang Leting and his “Lao Shi Zhen” prescription is used to illustrate how the model operates in practice. Conclusion: The study provides a systematic perspective for understanding medical experts' tacit knowledge and offers theoretical insights for medical education, knowledge transmission, and clinical decision support.