LLMs are trained using many thousands of GPUs in well-known conventional models. It is necessary to address numerous issues in the training process, such as manual data collection organization, data parallel, model parallel, evaluation, testing, deployment, transferring large data streams, detecting errors, ongoing maintenance, and project management. A team of dozens of engineers is required to handle system problems in the training process. Therefore, it is time-consuming and expensive to build an efficient and fault-tolerant system based on Kubernetes. This paper develops SimpleScale for building LLMs based on FSDP and Slurm, which is a simple and efficient training system that includes the training agent, the efficient parallel strategy, the optimal step of checkpoint, and so on. Using the proposed system enables us to significantly accelerate the process of building the LLM without incurring substantial time spent on various manual issues. The proposed 1024-GPU cluster was tested on TinyLlama, which has 1.1 billion parameters and 300 billion tokens. For example, utilizing a 16-H100 GPU cluster accelerated the traditional TinyLlama training time costs from 89.05 days to 39.05 days. In the future, the project team plans to integrate Flash-Attention3, aiming for an MFU of more than 60% while maintaining the acceleration achieved in the present work.

Joint torque distribution has been the focus of the electrically actuated quadruped robot since the appearance of this work. Most approaches have focused on the spring-loaded-inverted-pendulum (SLIP) model. This work aims to develop a joint torque redistribution method for each joint to perform a hopping movements for the single legs. By investigating the variation of momentum and the energy consumption for quadruped robots, the work presents the joint torque redistribution model. The proposed approach is utilized to plan a periodic trajectory movement for quadruped robots. The stability and energy efficiency of the proposed algorithm are verified with the actual single leg robot prototype.