General-purpose Sparse Matrix-Matrix Multiplication (SpMM) is a fundamental kernel in scientific computing and deep learning. The emergence of new matrix computation units such as Tensor Cores (TCs) brings more opportunities for SpMM acceleration. However, in order to fully unleash the power of hardware performance, systematic optimization is required. In this paper, we propose Acc-SpMM, a high-performance SpMM library on TCs, with multiple optimizations, including data-affinity-based reordering, memory efficient compressed format, high-throughput pipeline, and adaptive sparsity-aware load balancing. In contrast to the state-of-the-art SpMM kernels on various NVIDIA GPU architectures with a diverse range of benchmark matrices, Acc-SpMM achieves significant performance improvements, on average 2.52x (up to 5.11x) speedup on RTX 4090, on average 1.91x (up to 4.68x) speedup on A800, and on average 1.58x (up to 3.60x) speedup on H100 over cuSPARSE.

Mixture-of-Experts (MoE) efficiently trains large models by using sparse activation to lower costs, selecting a few experts based on data characteristics. However, it faces challenges such as All-to-All communication overhead and load imbalance, with most optimizations targeting dynamic graphs rather than the more efficient static graphs. This study identifies two key challenges in training MoE on static graphs: 1) excessive All-to-All communication (up to 75% of iteration time) and load imbalance (70% of tokens handled by two experts) between experts due to the sparse structure of the MoE model and the token distribution; and 2) inefficient zero-padding for static shapes, leading to unnecessary computational overhead(wasting approximately 50% of resources). Thus, EfficientMoE, a scheduling method based on expert load and data characteristics, is introduced. EfficientMoE first designs a sampler to collect real-time information about token distribution, expert load, etc. It constructs a load prediction model to evaluate expert load. Subsequently, EfficientMoE proposes a dynamic schedule strategy for experts with evaluated expert load, reducing All-to-All communication and addressing load-balancing issues. Additionally, an expert capacity model is proposed to set different capacities for replicas of hot experts before static graph compilation, minimizing computation and storage overhead caused by significant padding. This study implements EfficientMoE in MindSpore and uses 32 Ascend AI accelerators to train an MoE model with 21 billion parameters and evaluate its validity. EfficientMoE demonstrated an improvement of 30% in model training time, approximately 12% reduction in communication time, and saved 35% computational resources across different clusters, compared with Switch transformers, and the Fastermoe method for static graphs.

Federated learning is a distributed machine learning method for learning consumer data generated by consumer electronic devices. It provides personalized intelligent services for users while maintaining data privacy by sharing models instead of user data. It is challenging for these methods to learn data efficiently and with high quality. The performance and quality of the model in federated learning are affected by differences in data from different devices, which can lead to divergence between local and global models. Therefore, this paper proposes a parameter importance-based optimization federated learning method, FedPIA. First, two key indicators, including the rate of parameter change and the acceleration rate of parameter change, are provided to evaluate the importance of model parameters, which can reflect the divergence of the data from different devices. Then, a regular term is created based on these indicators to construct a new loss function, reducing the divergence between local and global models. Last, this paper efficiently trains a high-quality model on consumer data with this optimization approach. Experiments show that FedPIA is significantly better than FedCL, FedAvg, and Scaffold in accuracy, F1 scores and communication overhead.

An intelligent maritime navigation system is expected to play an important role in the realm of Internet of Vessels (IoV). As a key technology in navigation systems, vessel trajectory prediction technology is critical to the IoV. Automatic identification system (AIS), an automated tracking system, is used extensively for vessel trajectory prediction. However, certain characteristics in the AIS data, such as the large number of anchored trajectories in the area, anomalous sharp turns of some trajectories, and the behavioral differences of vessels in different segments, limit the prediction accuracy. In this study, we propose a novel vessel trajectory prediction model for accurate prediction with the following characteristics: (1) an anchor trajectory elimination algorithm to eliminate anchor trajectories; (2) a statistical trajectory restoration algorithm to repair sharp turning; (3) a two-stage clustering algorithm (D-KMEANS) to distinguish vessel behavior; and (4) a deep bidirectional gate recurrent unit (Stacked-BiGRUs) model to predict vessel trajectory and compare the accuracy of the model before and after improvement. The results show that the mean square error and the mean absolute error of the improved model are reduced by 27% and 46%, respectively. This research shows good potential for maritime navigation early warning and safety.