This study investigates the high-quality data processing technology, immersive experience mechanisms, and large-scale access in the Metaverse, concurrently ensuring robust privacy and security. We commence with a comprehensive analysis of the Metaverse’s service requirements, followed by an exploration of its principal technologies. Furthermore, we evaluate the feasibility and potential benefits of integrating semantic communication to enhance the service quality of the Metaverse. A federated semantic communication framework is proposed, integrating semantic data transmission, semantic digital twins, and a Metaverse construction model trained through federated learning. We proceed to assess the performance of our proposed framework through simulations, highlighting the notable enhancements in transmission efficiency, recovery effectiveness, and intelligent recognition ability afforded by semantic communication for the Metaverse. Notably, the framework achieves outstanding compression efficiency with minimal information distortion (0.055), which decreases transmission delays and improves the immersion quality within the Metaverse. Finally, we identify future challenges and propose potential solutions for advancing semantic communication, federated learning, and Metaverse technologies.
10 July 2024 in International Journal of Neural Systems
This paper addresses the task of semantic segmentation in computer vision, aiming to achieve precise pixel‐wise classification. We investigate the joint training of models for semantic edge detection and semantic segmentation, which has shown a promise. However, implicit cross‐task consistency learning in multitask networks is limited. To address this, we propose a novel “decoupled cross‐task consistency loss” that explicitly enhances cross‐task consistency. Our semantic segmentation network, TriangleNet, achieves a substantial 2.88% improvement over the Baseline in mean Intersection over Union (mIoU) on the Cityscapes test set. Notably, TriangleNet operates at 77.4% mIoU/46.2 FPS on Cityscapes, showcasing real‐time inference capabilities at full resolution. With multiscale inference, performance is further enhanced to 77.8%. Furthermore, TriangleNet consistently outperforms the Baseline on the FloodNet dataset, demonstrating its robust generalization capabilities. The proposed method underscores the significance of multitask learning and explicit cross‐task consistency enhancement for advancing semantic segmentation and highlights the potential of multitasking in real‐time semantic segmentation.
With the development of large language models (LLMs), they have demonstrated powerful capabilities across many downstream tasks. Existing Tibetan named entity recognition (NER) methods often suffer from a high degree of coupling between data and models, limiting them to identifying entities only within specific domain datasets and making cross-domain recognition difficult. Additionally, each dataset requires training a dedicated model, and when faced with new domains, retraining and redeployment are necessary. In practical applications, the ability to perform cross-domain NER is crucial to meeting real-world needs. To address this issue and decouple data from models, enabling cross-domain NER, this paper proposes a cross-domain joint learning approach based on large language models, which enhances model robustness by learning the shared underlying semantics across different domains. To reduce the significant computational costs incurred by LLMs during inference, we adopt an adaptive structured pruning method based on domain-dependent prompt, which effectively reduces the model’s memory requirements and improves the inference speed while minimizing the impact on performance. The experimental results show that our method significantly outperformed the baseline model across cross-domain Tibetan datasets. In the Tibetan medicine domain, our method achieved an F1 score improvement of up to 27.26% compared with the baseline model at its best. Our method achieved an average F1 score of 95.17% across domains, outperforming the baseline Llama2 + Prompt model by 5.12%. Furthermore, our method demonstrates strong generalization capabilities in NER tasks for other low-resource languages.