In this letter, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal waypoint is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot between waypoints towards a global goal. The fully autonomous navigation is performed without any prior knowledge while a map is recorded as the robot moves through the environment. Experiments show that the proposed method has an advantage over similar exploration methods, without reliance on a map or prior information in complex static as well as dynamic environments.

In this paper, we propose a goal-oriented obstacle avoidance navigation system based on deep reinforcement learning that uses depth information in scenes, as well as goal position in polar coordinates as state inputs. The control signals for robot motion are output in a continuous action space. We devise a deep deterministic policy gradient network with the inclusion of depth-wise separable convolution layers to process the large amounts of sequential depth image information. The goal-oriented obstacle avoidance navigation is performed without prior knowledge of the environment or a map. We show that through the proposed deep reinforcement learning network, a goal-oriented collision avoidance model can be trained end-to-end without manual tuning or supervision by a human operator. We train our model in a simulation, and the resulting network is directly transferred to other environments. Experiments show the capability of the trained network to navigate safely around obstacles and arrive at the designated goal positions in the simulation, as well as in the real world. The proposed method exhibits higher reliability than the compared approaches when navigating around obstacles with complex shapes. The experiments show that the approach is capable of avoiding not only static, but also dynamic obstacles.

In this paper, a Deep Reinforcement Learning (DRL)-based approach for learning mobile cleaning robot navigation commands that leverage experience from expert demonstrations is presented. First, expert demonstrations of robot motion trajectories in simulation in the cleaning robot domain are collected. The relevant motion features with regard to the distance to obstacles and the heading difference towards the navigation goal are extracted. Each feature weight is optimized with respect to the collected data, and the obtained values are assumed as representing the optimal motion of the expert navigation. A reward function is created based on the feature values to train a policy with semi-supervised DRL, where an immediate reward is calculated based on the closeness to the expert navigation. The presented results show the viability of this approach with regard to robot navigation as well as the reduced training time.

With the increase in the capabilities of robotic devices, there is a growing need for accurate and relevant environment maps. Current robotic devices can map their surrounding environment using a multitude of sensors as mapping sources. The challenge lies in combining these heterogeneous maps into a single, informative map to enhance the robustness of subsequent robot control algorithms. In this paper, we propose to perform map fusion as a post-processing step based on the alignment of the window of interest (WOI) from occupancy grid histograms. Initially, histograms are obtained from map pixels to determine the relevant WOI. Subsequently, they are transformed to align with a selected base image using the Manhattan distance of histogram values and the rotation angle from WOI line regression. We demonstrate that this method enables the combination of maps from multiple sources without the need for sensor calibration.

For mobile cleaning robot navigation, it is crucial to not only base the motion decisions on the ego agent’s capabilities but also to take into account other agents in the shared environment. Therefore, in this paper, we propose a deep reinforcement learning (DRL)-based approach for learning motion policy conditioned not only on ego observations of the environment, but also on incoming information about other agents. First, we extend a replay buffer to collect state observations on all agents at the scene and create a simulation setting from which to gather the training samples for DRL policy. Next, we express the incoming agent information in each agent’s frame of reference, thus making it translation and rotation invariant. We propose a neural network architecture with edge embedding layers that allows for the extraction of incoming information from a dynamic range of agents. This allows for generalization of the proposed approach to various settings with a variable number of agents at the scene. Through simulation results, we show that the introduction of edge layers improves the navigation policies in shared environments and performs better than other state-of-the-art DRL motion policy methods.

In this paper, we present an autonomous navigation system for goal-driven exploration of unknown environments through deep reinforcement learning (DRL). Points of interest (POI) for possible navigation directions are obtained from the environment and an optimal waypoint is selected, based on the available data. Following the waypoints, the robot is guided towards the global goal and the local optimum problem of reactive navigation is mitigated. Then, a motion policy for local navigation is learned through a DRL framework in a simulation. We develop a navigation system where this learned policy is integrated into a motion planning stack as the local navigation layer to move the robot between waypoints towards a global goal. The fully autonomous navigation is performed without any prior knowledge while a map is recorded as the robot moves through the environment. Experiments show that the proposed method has an advantage over similar exploration methods, without reliance on a map or prior information in complex static as well as dynamic environments.