The field of robotics has been rapidly developing in recent years, and the work related to training robotic agents with reinforcement learning has been a major focus of research. This survey reviews the application of reinforcement learning for pick-and-place operations, a task that a logistics robot can be trained to complete without support from a robotics engineer. To introduce this topic, we first review the fundamentals of reinforcement learning and various methods of policy optimization, such as value iteration and policy search. Next, factors which have an impact on the pick-and-place task, such as reward shaping, imitation learning, pose estimation, and simulation environment are examined. Following the review of the fundamentals and key factors for reinforcement learning, we present an extensive review of all methods implemented by researchers in the field to date. The strengths and weaknesses of each method from literature are discussed, and details about the contribution of each manuscript to the field are reviewed. The concluding critical discussion of the available literature, and the summary of open problems indicates that experiment validation, model generalization, and grasp pose selection are topics that require additional research.
The majority of robots in factories today are operated with conventional control strategies that require individual programming on a task-by-task basis, with no margin for error. As an alternative to the rudimentary operation planning and task-programming techniques, machine learning has shown significant promise for higher-level task planning, with the development of reinforcement learning (RL)-based control strategies. This paper reviews the implementation of combined traditional and RL control for simulated and real environments to validate the RL approach for standard industrial tasks such as reach, grasp, and pick-and-place. The goal of this research is to bring intelligence to robotic control so that robotic operations can be completed without precisely defining the environment, constraints, and the action plan. The results from this approach provide optimistic preliminary data on the application of RL to real-world robotics.
Railway defects can result in substantial economic and human losses. Among all defects, surface defects are the most common and prominent type, and various optical-based non-destructive testing (NDT) methods have been employed to detect them. In NDT, reliable and accurate interpretation of test data is vital for effective defect detection. Among the many sources of errors, human errors are the most unpredictable and frequent. Artificial intelligence (AI) has the potential to address this challenge; however, the lack of sufficient railway images with diverse types of defects is the major obstacle to training the AI models through supervised learning. To overcome this obstacle, this research proposes the RailGAN model, which enhances the basic CycleGAN model by introducing a pre-sampling stage for railway tracks. Two pre-sampling techniques are tested for the RailGAN model: image-filtration, and U-Net. By applying both techniques to 20 real-time railway images, it is demonstrated that U-Net produces more consistent results in image segmentation across all images and is less affected by the pixel intensity values of the railway track. Comparison of the RailGAN model with U-Net and the original CycleGAN model on real-time railway images reveals that the original CycleGAN model generates defects in the irrelevant background, while the RailGAN model produces synthetic defect patterns exclusively on the railway surface. The artificial images generated by the RailGAN model closely resemble real cracks on railway tracks and are suitable for training neural-network-based defect identification algorithms. The effectiveness of the RailGAN model can be evaluated by training a defect identification algorithm with the generated dataset and applying it to real defect images. The proposed RailGAN model has the potential to improve the accuracy of NDT for railway defects, which can ultimately lead to increased safety and reduced economic losses. The method is currently performed offline, but further study is planned to achieve real-time defect detection in the future.