23 January 2025 in Knowledge-Based Systems
1 January 2023 in Journal of Artificial Intelligence Practice
With the development of cloud computing and the continuous development of infrastructure, architecture upgrades and other technologies, Baidu's internal services are gradually moving to the cloud environment. Although the efficiency of services has increased significantly, some shortcomings and deficiencies of the basic capabilities of the cloud environment have gradually become apparent, resulting in the inability to meet some reasonable requirements of the enterprise, such as building the topology relationship between different microservices and conducting the test session for The traditional way of implementation is to record the real traffic to reflect and verify the function and so on. The traditional way of implementation is often to implant the code into the business system to make changes. However, given the diversity of business forms and technologies, the conventional way has a lot of problems in terms of business intervention, communication and coordination, performance, stability, and other aspects. In this paper, we introduce Baidu's eBPF-based network framework: DeeTune, which provides the ability to create service topology, record traffic, monitor non-intrusive metrics, etc., further improve the efficiency of SRE and quality assurance.
28 May 2026 in IEEE Transactions on Audio, Speech and Language Processing
Large Language Models (LLMs) have shown great promise in multilingual machine translation (MT), even with limited bilingual supervision. However, fine-tuning LLMs with parallel corpora presents major challenges, namely parameter interference. To address these issues, we propose Mix-MoE, a mixed Mixture-of-Experts framework designed to train LLMs for multilingual MT. Our framework operates in two distinct stages: (1) post-pretraining with MoE on monolingual corpora, and (2) post-pretraining with MoE on parallel corpora. Crucially, we divide the MoE layers into two specialized groups: Language Model Experts (LM Experts) and Machine Translation Experts (MT Experts). LM Experts are designed to capture and retain the monolingual knowledge learned by the pre-trained LLM. MT Experts, on the other hand, are specifically trained to acquire and store bilingual translation knowledge. Furthermore, to facilitate effective interaction between these specialized experts and leverage potential underlying structural patterns in text, we introduce a routing mechanism enhanced by Fourier Transform features derived from model representations. The experimental results demonstrate that Mix-MoE excels in multilingual MT, significantly outperforming existing baselines and showing notable progress in mitigating parameter interference.