The integration of Artificial Intelligence (AI) with the Internet of Things (IoT), known as the Artificial Intelligence of Things (AIoT), enhances the devices’ processing and analysis capabilities and disrupts such sectors as healthcare, industry, and oil. However, AIoT’s complexity and scale are challenging for traditional machine learning (ML). Deep learning offers a solution but has limited testability, verifiability, and interpretability. In turn, theneuro-symbolic paradigmaddresses these challenges by combining the robustness of symbolic AI with the flexibility of DL, enabling AI systems to reason, make decisions, and generalize knowledge from large datasets better. This paper reviews state-of-the-art DL models for IoT, identifies their limitations, and explores how neuro-symbolic methods can overcome them. It also discusses key challenges and research opportunities in enhancing AIoT reliability with neuro-symbolic approaches, including hard-coded symbolic AI, multimodal sensor data, biased interpretability, trading-off interpretability, and performance, complexity in integrating neural networks and symbolic AI, and ethical and societal challenges.
27 June 2020 in Scalable Computing: Practice and Experience
The application of deep learning in industry often needs to train large-scale neural networks and use large-scale data sets. However, larger networks and larger data sets lead to longer training time, which hinders the research of algorithms and the progress of actual engineering development. Data-parallel distributed training is a commonly used solution, but it is still in the stage of technical exploration. In this paper, we study how to improve the training accuracy and speed of distributed training, and propose a distributed training strategy based on hybrid gradient computing. Specifically, in the gradient descent stage, we propose a hybrid method, which combines a new warmup scheme with the linear-scaling stochastic gradient descent (SGD) algorithm to effectively improve the training accuracy and convergence rate. At the same time, we adopt the mixed precision gradient computing. In the single-GPU gradient computing and inter-GPU gradient synchronization, we use the mixed numerical precision of single precision (FP32) and half precision (FP16), which not only improves the training speed of single-GPU, but also improves the speed of inter-GPU communication. Through the integration of various training strategies and system engineering implementation, we finished ResNet-50 training in 20 minutes on a cluster of 24 V100 GPUs, with 75.6% Top-1 accuracy, and 97.5% GPU scaling efficiency. In addition, this paper proposes a new criterion for the evaluation of the distributed training efficiency, that is, the actual average single-GPU training time, which can evaluate the improvement of training methods in a more reasonable manner than just the improved performance due to the increased number of GPUs. In terms of this criterion, our method outperforms those existing methods.