This paper develops a navigation strategy based on deep reinforcement learning (DRL) for mobile robots. Because of the large difference between simulation and reality, most of the trained DRL models cannot be directly migrated into real robots. Moreover, how to explore in a sparsely rewardedenvironment isalsoalong-standing problemof DRL. This study proposes an end-to-end navigation planner that translates sparse laser ranging results into movement actions. Using this highly abstract data as input, agents trained by simulation can be extended to the real scene for practical application. For map-less navigation across obstacles and traps, it is difficult to reach the target via random exploration. Curiosity is used to encourage agents to explore the state of an environment that has not been visited and as an additional reward for exploring behavior. The agent relies on the self-supervised model to predict the next state, based on the current state and the executed action. The prediction error is used as a measure of curiosity. The experimental results demonstrate that without any manual design features and previous demonstrations, the proposed method accomplishes map-less navigation in complex environments. Through a reward signal that is enhanced by intrinsic motivation, the agent explores more efficiently and the learned strategy is more reliable.

Image-based visual servoing (IBVS) can reach a desired position for a relatively stationary target using continuous visual feedback. Proper feature extraction and appropriate servoing control laws are essential to performance for IBVS. IBVS control can be interrupted or interfered abruptly if no features are extracted when the observed object is occluded. To address the problem of missing feature points in current images during a visual navigation task, a homography method that uses a priori visual information is proposed to predict all of the missing feature points and to ensure the execution of IBVS. The mixture parameter for the image Jacobian matrix can also affect the control of IBVS. The settings for the mixture parameter are heuristic so there is no a systematic approach for most IBVS applications. An adaptive control approach is proposed to determine the mixture parameter. The proposed method uses a reinforcement learning (RL) method to adaptively adjust the mixture parameter during the robot movement, which allows more efficient control than a constant parameter. A logarithmic interval state space partition for RL is used to ensure efficient learning. The integrated visual servoing control system is validated by several experiments that involve wheeled mobile robots reaching a target with a desired configuration. The results for simulation and experiment demonstrate that the proposed method has a faster convergence rate than other methods.

Some researchers have introduced transfer learning mechanisms to multiagent reinforcement learning (MARL). However, the existing works devoted to cross-task transfer for multiagent systems were designed just for homogeneous agents or similar domains. This work proposes an all-purpose cross-transfer method, called multiagent lateral transfer (MALT), assisting MARL with alleviating the training burden. We discuss several challenges in developing an all-purpose multiagent cross-task transfer learning method and provide a feasible way of reusing knowledge for MARL. In the developed method, we take features as the transfer object rather than policies or experiences, inspired by the progressive network. To achieve more efficient transfer, we assign pretrained policy networks for agents based on clustering, while an attention module is introduced to enhance the transfer framework. The proposed method has no strict requirements for the source task and target task. Compared with the existing works, our method can transfer knowledge among heterogeneous agents and also avoid negative transfer in the case of fully different tasks. As far as we know, this article is the first work denoted to all-purpose cross-task transfer for MARL. Several experiments in various scenarios have been conducted to compare the performance of the proposed method with baselines. The results demonstrate that the method is sufficiently flexible for most settings, including cooperative, competitive, homogeneous, and heterogeneous configurations.

Robotic soccer games, which have become popular, require timely and precise decisionmaking in a dynamic environment. To address the problems of complexity in a critical situation, policy improvement in robotic soccer games must occur. This paper proposes an adaptive decisionmaking method that uses reinforcement learning (RL), and the decision-making system for a robotic soccer game is composed of two subsystems. The first subsystem in the architecture for the proposed method criticizes the situation, and the second subsystem implements decision-making policy. Inspired by the support vector machine (SVM), a situation classification method, which is called an improved SVM, embeds a decision tree structure and simultaneously addresses the problems of a large scale and multiple classifications. When a variety of situations that are collected in the field are classified and congregated into the tree structure, the problem of local strategy selection for each individual class of situations over time is regarded as a RL problem and is solved using a Q-learning method. The results of simulations and experiments demonstrate that the proposed method allows satisfactory decision-making.