Structured light calibration and robot handeye calibration are two of the most crucial parts in welding robot laser vision systems. The main aspect of this calibration is accuracy. To reduce the impact of errors in the calibration process and improve accuracy, an end-to-end calibration method for laser vision systems based on deep reinforcement learning is proposed. The proposed method involves two dual learning tasks: a pixel-to-point module and a point-to-wrist transformation. The pixel-to-point module predicts the locations of the feature points in the welding image, while the point-to-wrist transformation establishes an accurate conversion relationship from the pixel frame to the wrist frame of the robot. Point-to-wrist transformation consists of two major components: ‘actor’ and ‘critic’. The ‘actor’ model aims to infer the coordinates of the feature points in the wrist frame of the robot. In offline training, the ‘critic’ model is introduced to guide the learning process of the ‘actor’ model by maintaining geometric consistency between the image coordinate and robot coordinate. According to experimental results from the welding robot with continuous motion and changing poses, the proposed method significantly reduces the impact of errors in the calibration process and establishes a more accurate conversion relationship.

Multimodal transportation is a modern way of cargo transportation. With the increasing demand for cargo transportation, higher requirements are being placed on multimodal transportation multi-objective routing optimization. In multimodal transportation multi-objective routing optimization, in response to the limitations of classical algorithms in solving large-scale problems with multiple nodes and modes of transport, the limitations of directed transportation networks in the application, and the uncertainty of transport time, this paper proposes an optimization framework based on multi-objective weighted sum Q-learning, combined with the proposed undirected multiple-node network, and characterizes the uncertainty of time with a positively skewed distribution. The undirected multiple-node transportation network can better simulate cargo transportation and characterize transfer information, facilitate the modification of origin and destination, and avoid suboptimal solutions due to the manual setting of wrong route directions. The network is combined with weighted sum Q-learning to solve multimodal transportation multi-objective routing optimization problems faster and better. When modeling the uncertainty of transport time, a positively skewed distribution is used. The three objectives of transport cost, carbon emission cost, and transport time were studied and compared with PSO, GA, AFO, NSGA-II, and MOPSO. The experimental results show that compared with PSO, GA, and AFO using a directed transportation network, the proposed method has a significant improvement in optimization results and running time, and the running time is shortened by 26 times. The proposed method can better solve the boundary of the Pareto front and dominate the partial solutions of NSGA-II and MOPSO. The effect of time uncertainty on the performance of the algorithm is more significant in transport orders with high time weight. With the increase in uncertainty, the reliability of the route decreases. The effectiveness of the proposed method is verified.