In recent years, intelligent fault diagnosis technology with the deep learning algorithm has been widely used in the manufacturing industry for substituting time-consuming human analysis method to enhance the efficiency of fault diagnosis. The rolling bearing as the connection between the rotor and support is the crucial component in rotating equipment. However, the working condition of the rolling bearing is under changing with complex operation demand, which will significantly degrade the performance of the intelligent fault diagnosis method. In this paper, a new deep transfer model based on Wasserstein Distance guided Multi-Adversarial Networks (WDMAN) is proposed to address this problem. The WDMAN model exploits complex feature space structures to enable transfer of different data distributions based on multiple domain critic networks. The essence of our method is learning the shared feature representation by minimizing the Wasserstein distance between the source domain and target domain distribution in an adversarial training way. Experiment results demonstrate that our model outperforms the state-of-the-art methods on rolling bearing fault diagnosis under different working conditions. The t-Distributed Stochastic Neighbor Embedding (t-SNE) technology is used to visualize the learned domain invariant feature and investigate the transferability behind the great performance of our proposed model.

Current constrained reinforcement learning (RL) methods guarantee constraint satisfaction only in expectation, which is inadequate for safety-critical decision problems. Since a constraint satisfied in expectation remains a high probability of exceeding the cost threshold, solving constrained RL problems with high probabilities of satisfaction is critical for RL safety. In this work, we consider the safety criterion as a constraint on the conditional value-at-risk (CVaR) of cumulative costs, and propose the CVaR-constrained policy optimization algorithm (CVaR-CPO) to maximize the expected return while ensuring agents pay attention to the upper tail of constraint costs. According to the bound on the CVaR-related performance between two policies, we first reformulate the CVaR-constrained problem in augmented state space using the state extension procedure and the trust-region method. CVaR-CPO then derives the optimal update policy by applying the Lagrangian method to the constrained optimization problem. In addition, CVaR-CPO utilizes the distribution of constraint costs to provide an efficient quantile-based estimation of the CVaR-related value function. We conduct experiments on constrained control tasks to show that the proposed method can produce behaviors that satisfy safety constraints, and achieve comparable performance to most safe RL (SRL) methods.

With the breakthrough of AlphaGo, deep reinforcement learning has become a recognized technique for solving sequential decision-making problems. Despite its reputation, data inefficiency caused by its trial and error learning mechanism makes deep reinforcement learning difficult to apply in a wide range of areas. Many methods have been developed for sample efficient deep reinforcement learning, such as environment modelling, experience transfer, and distributed modifications, among which distributed deep reinforcement learning has shown its potential in various applications, such as human-computer gaming and intelligent transportation. In this paper, we conclude the state of this exciting field, by comparing the classical distributed deep reinforcement learning methods and studying important components to achieve efficient distributed learning, covering single player single agent distributed deep reinforcement learning to the most complex multiple players multiple agents distributed deep reinforcement learning. Furthermore, we review recently released toolboxes that help to realize distributed deep reinforcement learning without many modifications of their non-distributed versions. By analysing their strengths and weaknesses, a multi-player multi-agent distributed deep reinforcement learning toolbox is developed and released, which is further validated on Wargame, a complex environment, showing the usability of the proposed toolbox for multiple players and multiple agents distributed deep reinforcement learning under complex games. Finally, we try to point out challenges and future trends, hoping that this brief review can provide a guide or a spark for researchers who are interested in distributed deep reinforcement learning.

With the breakthrough of AlphaGo, human-computer gaming AI has ushered in a big explosion, attracting more and more researchers all over the world. As a recognized standard for testing artificial intelligence, various human-computer gaming AI systems (AIs) have been developed, such as Libratus, OpenAI Five, and AlphaStar, which beat professional human players. The rapid development of human-computer gaming AIs indicates a big step for decision-making intelligence, and it seems that current techniques can handle very complex human-computer games. So, one natural question arises: What are the possible challenges of current techniques in human-computer gaming and what are the future trends? To answer the above question, in this paper, we survey recent successful game AIs, covering board game AIs, card game AIs, first-person shooting game AIs, and real-time strategy game AIs. Through this survey, we 1) compare the main difficulties among different kinds of games and the corresponding techniques utilized for achieving professional human-level AIs; 2) summarize the mainstream frameworks and techniques that can be properly relied on for developing AIs for complex human-computer games; 3) raise the challenges or drawbacks of current techniques in the successful AIs; and 4) try to point out future trends in human-computer gaming AIs. Finally, we hope that this brief review can provide an introduction for beginners and inspire insight for researchers in the field of AI in human-computer gaming.

As a key on-orbit service technology, relative measurement of non-cooperative spatial targets can give accurate relative pose for the unmanned rendezvous and docking of non-cooperative targets. With the rapid development of three-dimensional data acquisition equipment such as flash lidar in the field of non-cooperative targets’ pose measurement in recent years, research on the pose tracking technology based on the three-dimensional (3D) point cloud of non-cooperative targets has become more and more urgent. In this paper, we proposed an approach for autonomous recognition and pose tracking of non-cooperative targets based on point cloud features. For this method, the density, curvature and normal angle of a non-cooperative target 3D point cloud are first calculated as the local feature description (DCA) of the point cloud, and then the feature description is used to match and remove errors through RANSAC. Finally the Iterative Closest Point (ICP) algorithm is adopted to obtain the precise pose of the non-cooperative target. Experimental results have shown that this approach can effectively find the features of non-cooperative tumbling targets, get rid of the dependence on the target model and database, and realize the pose tracking of non-cooperative targets.

The objective of this study was to introduce a novel deep learning technique for more accurate children caries diagnosis on dental panoramic radiographs. Specifically, a swin transformer is introduced, which is compared with the state-of-the-art convolutional neural network (CNN) methods that are widely used for caries diagnosis. A tooth type enhanced swin transformer is further proposed by considering the differences among canine, molar and incisor. Modeling the above differences in swin transformer, the proposed method was expected to mine domain knowledge for more accurate caries diagnosis. To test the proposed method, a children panoramic radiograph database was built and labeled with a total of 6028 teeth. Swin transformer shows better diagnosis performance compared with typical CNN methods, which indicates the usefulness of this new technique for children caries diagnosis on panoramic radiographs. Furthermore, the proposed tooth type enhanced swin transformer outperforms the naive swin transformer with the accuracy, precision, recall, F1 and area-under-the-curve being 0.8557, 0.8832, 0.8317, 0.8567 and 0.9223, respectively. This indicates that the transformer model can be further improved with a consideration of domain knowledge instead of a copy of previous transformer models designed for natural images. Finally, we compare the proposed tooth type enhanced swin transformer with two attending doctors. The proposed method shows higher caries diagnosis accuracy for the first and second primary molars, which may assist dentists in caries diagnosis.