A long-standing goal of the Human-Robot Collaboration (HRC) in manufacturing systems is to increase the collaborative working efficiency. In line with the trend of Industry 4.0 to build up the smart manufacturing system, the collaborative robot in the HRC system deserves better designing to be more self-organized and to find the superhuman proficiency by self-learning. Inspired by the impressive machine learning algorithms developed by Google Deep Mind like Alphago Zero, in this paper, the human-robot collaborative assembly working process is formatted into a chessboard and the selection of moves in the chessboard is used to analogize the decision-making by both human and robot in the HRC assembly working process. To obtain the optimal policy of the working sequence to maximize the working efficiency, agents in the system are trained with a self-play algorithm based on reinforcement learning, without guidance or domain knowledge beyond game rules. A convolution neural network (CNN) is also trained to predict the distribution of the priority of move selections and whether a working sequence is the one resulting in the maximum of the HRC efficiency. A height-adjustable standing desk assembly is used to demonstrate the proposed HRC assembly algorithm and its efficiency in real-time task planning.

This letter aims at modeling and real-time control of a flexible manufacturing system (FMS) that is operated by multi-skilled mobile robots. By introducing the idea of a unique ideal clean configuration and the effective disruption event, the concepts of Opportunity Window (OW) and Permanent Production Loss (PPL) are extended to the FMS. The control considered in this letter is a robot assignment problem where individual robots are dynamically assigned to each workstation in real time to improve performance. The problem is formulated as a Markov Decision Process (MDP) and solved using the Double Deep Q-Network (DDQN) reinforcement learning algorithm. PPL is used as the reward setting for training and a space discretization is used between each workstation to capture real time movement of the robots on the plant floor. The effectiveness of the control strategy is then demonstrated by comparing the results to several heuristic control schemes.

Real time production performance evaluation plays a vital role in diagnosing manufacturing system health status and achieving productivity improvements. However, most existing studies on system performance evaluation are based on steady state analysis and focused on the production system with fixed cycle time machines. The real-time performance evaluation for a manufacturing system with variable cycle time machines, although typical for a large number of realistic scenarios, has been mostly ignored. The development of smart manufacturing and increasingly available sensor data have provided unprecedented opportunities to carry out thorough analysis on the real-time performance of such complex systems. In this letter, we developed a data-enabled methodology to efficiently identify and predict the real-time permanent production loss for a serial production line with variable cycle time machines. The concept and evaluation method of opportunity window are introduced to facilitate the permanent production loss estimation. Numerical case studies are presented to demonstrate the effectiveness of the proposed methods for opportunity window evaluation and production loss identification.

In the manufacturing industry, the random failures of machines introduce unexpected disruptions to the production. It is preferable to replace those aged machines with new ones before they fail. In this paper, the machine preventive replacement problem in serial production lines is discussed. To obtain an optimal machine replacement policy, the problem is formulated as a reinforcement learning problem and solved with the Q-learning algorithm. A reward function is proposed based on the production loss evaluation of the serial production lines. The data-driven modeling of serial production lines is used during the training process of the agent. A simulation study is conducted to evaluate the efficiency and effectiveness of the proposed method.