Reinforcement Learning (RL) can be considered as a sequence modeling task, where an agent employs a sequence of past state-action-reward experiences to predict a sequence of future actions. In this work, we propose St ate- A ction- R eward Transformer ( StAR former), a Transformer architecture for robot learning with image inputs, which explicitly models short-term state-action-reward representations (StAR-representations), essentially introducing a Markovian-like inductive bias to improve long-term modeling. StARformer first extracts StAR-representations using self-attending patches of image states, action, and reward tokens within a short temporal window. These StAR-representations are combined with pure image state representations, extracted as convolutional features, to perform self-attention over the whole sequence. Our experimental results show that StARformer outperforms the state-of-the-art Transformer-based method on image-based Atari and DeepMind Control Suite benchmarks, under both offline-RL and imitation learning settings. We find that models can benefit from our combination of patch-wise and convolutional image embeddings. StARformer is also more compliant with longer sequences of inputs than the baseline method. Finally, we demonstrate how StARformer can be successfully applied to a real-world robot imitation learning setting via a human-following task.

This paper presents a method for building a probability grid map for autonomous mobile robots with ultrasonic sensors using a data association filter (DAF). The method is based on evaluating the possibility that the acquired sonar data are all reflected by the same object. The DAF is able to associate data points with each other. Data affected by specular reflection are not likely to be associated with the same object, so they are excluded from the data cluster by the DAF, thereby improving the reliability of the data used for the probability grid map. Since the corrupted data are not used to update the probability map, it is possible to build a good quality grid map even in a specular environment. The DAF was applied to the Bayesian and the Orientation probability models, which are typical models used to build grid maps, to verify its effectiveness. Experimental results were also obtained using a mobile robot in a real-world environment.

A mobile robot must be able to build a reliable map of surroundings and estimate its position. We have developed a technique for a grid-based localization of a mobile robot with ultrasonic sensors using extended Kalman filter (EKF). For this, we used grids themselves as landmarks of the environment. The grid-based localization can minimize the use of computer resources for localization because this approach does not rely on exact geometric representation of a landmark. Experiments were performed in a real environment to verify the methodology developed in this study, and the results indicate that the grid-based localization can be useful for a practical application.

This paper presents a technique for accurate localization of mobile robots using an enhanced topological map and using the low-cost sensors such as wheel odometer, global positioning system (GPS), and mono-camera. The localization framework is based on EKF to fuse the sensor data and the topological map. The sensor data include the positions of traffic marks measured by camera and topological map having the actual positions of traffic marks extracted from aerial or satellite images in advance. Our approach obtains the adaptive parameter for EKF localization by matching two positions, measured by camera and extracted from topological map, on each traffic mark. The adaptive parameter reflects the geographical characteristics, e.g. hill, corner, and road surfaces. The proposed method has shown high accuracy result and apparently better performance of the EKF localization with adaptive parameter. The proposed method is economically feasible and practically applicable to commercial robots using the low-cost sensors and providing the reliable localization services.