The ability to automate machine translation has various applications in international commerce, medicine, travel, education, and text digitization. Due to the different grammar and lack of clear word boundaries in Chinese, it is challenging to conduct translation from word-based languages (e.g., English) to Chinese. This article has implemented a GPU-enabled deep learning machine translation system based on a domain-specific corpus. Our system takes English text as input and uses an encoder-decoder model with an attention mechanism based on Google’s Transformer to translate the text to Chinese output. The model was trained using a simple self-designed entropy loss function and an Adam optimizer on English–Chinese bilingual text sentences from the News area of the UM-Corpus. The parallel training process of our model can be performed on common laptops, desktops, and servers with one or more GPUs. At training time, we not only track loss over training epochs but also measure the quality of our model’s translations with the BLEU score. We also provide an easy-to-use web interface for users so as to manage corpus, training projects, and trained models. The experimental results show that we can achieve a maximum BLEU score of 29.2. We can further improve this score by tuning other hyperparameters. The GPU-enabled model training runs over 15x faster than on a multi-core CPU, which facilitates us having a shorter turn-around time. As a case study, we compare the performance of our model to that of Baidu’s, which shows that our model can compete with the industry-level translation system. We argue that our deep-learning-based translation system is particularly suitable for teaching purposes and small/medium-sized enterprises.
The BERT model is regarded as the cornerstone of various pre-trained large language models that have achieved promising results in recent years. This article investigates how to optimize the BERT model in terms of fine-tuning speed and prediction accuracy, aiming to accelerate the execution of the BERT model on a multi-core processor and improve its prediction accuracy in typical downstream natural language processing tasks. Our contributions are two-fold. First, we port and parallelize the fine-tuning training of the BERT model on a multi-core shared-memory processor. We port the BERT model onto a multi-core processor platform to accelerate the fine-tuning training process of the model for downstream tasks. Second, we improve the prediction performance of typical downstream natural language processing tasks through fine-tuning the model parameters. We select five typical downstream natural language processing tasks (CoLA, SST-2, MRPC, RTE, and WNLI) and perform optimization on the multi-core platform, taking the hyperparameters of batch size, learning rate, and training epochs into account. Our experimental results show that, by increasing the number of CPUs and the number of threads, the model training time can be significantly reduced. We observe that the reduced time is primarily concentrated in the self-attention mechanism. Our further experimental results show that setting reasonable hyperparameters can improve the accuracy of the BERT model when applied to downstream tasks and that appropriately increasing the batch size under conditions of sufficient computing resources can significantly reduce training time.
LLM-based language services (e.g., summarization, translation, and question answering) require low-latency and cost-effective inference, yet GPU resources may be costly, unavailable, or difficult to integrate in many deployments. Many-core CPUs provide a flexible alternative, but achieving efficient inference requires optimizations that match the distinct computational characteristics of the Prefill and Decode stages. This paper presents a stage-aware optimization approach for Llama 3 inference on two Phytium many-core processors (FT-2000+/64 and FT-1500A). Fine-grained profiling shows that matrix-vector multiplication (MVM) in the Decode stage accounts for 99.4% of Decode time, whereas Prefill exposes additional token-level parallelism. Accordingly, in Prefill we reorganize multiple token-wise MVM operations into matrix–matrix multiplication and select the GEMM implementation by micro-batch granularity, including a blocked C GEMM (up to over token-wise MVM) and a LibShalom-based tile path. In Decode, we apply OpenMP-based intra-kernel parallelism to MVM, RMSNorm, Softmax, and attention-head computation. Experiments on representative GLUE workloads show that the self-kernel implementation improves time per output token (TPOT) by at two threads ( at eight). A complete baseline study then shows that a tuned BLAS GEMV accelerates the dominant Decode MVM by a further ; following the stage-aware selection principle, the final configuration reaches a TPOT of 141.9 ms—a end-to-end speedup (7.05 tokens/s per 16-core node)—with scaling that continues to all cores. A kernel-level scalability study on Phytium FT-2000+/64 (Decode MVM up to at eight threads) further shows that saturation beyond eight threads is a property of the kernel implementation rather than of the platform. These results demonstrate that stage-aware characterization and per-stage implementation selection can substantially improve single-request Llama 3 inference efficiency on many-core CPUs, and provide guidance for CPU-side optimization in language-service deployments.