21 November 2019 in IEEE Access
As the volume of machine learning training data sets and the quantity of model parameters continue to grow, the pattern in which machine learning models are trained alone can no longer accommodate large-scale data environments. However, distributed systems and mobile edge computing systems are unpre-dictable and have heterogeneous nodes, resulting in interruptions in training or low convergence rate. In ad-dition, existing distributed machine learning frameworks cannot guarantee a good convergence rate and speedup ratio in a variety of operating environments. Considering the above shortcomings, this paper proposes an adaptive scheduling framework for machine learning based on a heterogeneous distributed system and mobile edge computing system for machine learning model optimization. The framework detects and analyzes the dynamic changes of resources in the distributed system and mobile edge computing system through the resource detection system; then, the task scheduling system adaptively modifies the environmental parameters and schedules calculations. Relevant experiments conducted with the public data set show that the robustness and scalability of the framework are significantly better than the traditional distributed machine learning framework under the premise of ensuring high convergence rate.
28 November 2019 in IEEE Access
With the application of mobile computing in the security field, security monitoring big data has also begun to emerge, providing favorable support for smart city construction and city-scale and investment expansion. Mobile computing takes full advantage of the computing power and communication capabilities of various sensing devices and uses these devices to form a computing cluster. When using such clusters for training of distributed machine learning models, the load imbalance and network transmission delay result in low efficiency of model training. Therefore, this paper proposes a distributed machine learning parameter communication consistency model based on the parameter server idea, which is called the limited synchronous parallel model. The model is based on the fault-tolerant characteristics of the machine learning algorithm, and it dynamically limits the size of the synchronization barrier of the parameter server, reduces the synchronization communication overhead, and ensures the accuracy of the model training; thus, the model realizes finite asynchronous calculation between the worker nodes and gives full play to the overall performance of the cluster. The implementation of cluster dynamic load balancing experiments shows that the model can fully utilize the cluster performance during the training of distributed machine learning models to ensure the accuracy of the model and improve the training speed.
25 November 2019 in IEEE Access
In distributed real-time machine learning of smart sensing equipment, training speed and training accuracy are two hard-to-choose trade-off performance measures directly influenced by the design of distributed machine learning algorithms. And it will influence effort of smart sensing equipment directly. We take the model aggregation method of distributed machine learning as a starting point. Due to the loss of accuracy caused by the direct averaging of the parameter average method, we developed the loss function weight reorder stochastic gradient descent method (LR-SGD). LR-SGD uses the loss function value to determine the weight of the work nodes when aggregating the model parameters, and it improves the performance of the parameter average method for nonconvex problems. As shown in the experiment results, our algorithm can improve the training accuracy by a maximum of approximately 0.57% for the Bulk Synchronous Parallel (BSP) model and approximately 6.30% for the Stale Synchronous Parallel (SSP) model.