The path following algorithm was proposed recently to approximately solve the matching problems on undirected graph models and exhibited a state-of-the-art performance on matching accuracy. In this paper, we extend the path following algorithm to the matching problems on directed graph models by proposing a concave relaxation for the problem. Based on the concave and convex relaxations, a series of objective functions are constructed, and the Frank-Wolfe algorithm is then utilized to minimize them. Several experiments on synthetic and real data witness the validity of the extended path following algorithm.

Policy search in reinforcement learning (RL) is a practical approach to interact directly with environments in parameter spaces, that often deal with dilemmas of local optima and real-time sample collection. A promising algorithm, known as guided policy search (GPS), is capable of handling the challenge of training samples using trajectory-centric methods. It can also provide asymptotic local convergence guarantees. However, in its current form, the GPS algorithm cannot operate in sequential multitask learning scenarios. This is due to its batch-style training requirement, where all training samples are collectively provided at the start of the learning process. The algorithm's adaptation is thus hindered for real-time applications, where training samples or tasks can arrive randomly. In this paper, the GPS approach is reformulated, by adapting a recently proposed, lifelong-learning method, and elastic weight consolidation. Specifically, Fisher information is incorporated to impart knowledge from previously learned tasks. The proposed algorithm, termed sequential multitask learning-GPS, is able to operate in sequential multitask learning settings and ensuring continuous policy learning, without catastrophic forgetting. Pendulum and robotic manipulation experiments demonstrate the new algorithms efficacy to learn control policies for handling sequentially arriving training samples, delivering comparable performance to the traditional, and batch-based GPS algorithm. In conclusion, the proposed algorithm is posited as a new benchmark for the real-time RL and robotics research community.

The task of Zero-Shot Object Goal Visual Navigation (ZSON) aims to enable robots to locate previously “unseen” objects by visual observations. This task presents a significant challenge since the robot must transfer the navigation policy learned from “seen” objects to “unseen” objects through auxiliary semantic information without training samples, a process known as zero-shot learning. In order to address this challenge, we propose a novel approach termed the Semantic Policy Network (SPNet). The SPNet consists of two modules that are deeply integrated with semantic embeddings: the Semantic Actor Policy (SAP) module and the Semantic Trajectory (ST) module. The SAP module generates actor network weight bias based on semantic embeddings, creating unique navigation policies for different target classes. The ST module records the robot's actions, visual features, and semantic embeddings at each step, and aggregates information in both the spatial and temporal dimensions. To evaluate our approach, we conducted extensive experiments using MP3D dataset, HM3D dataset and RoboTHOR. Experimental results indicate that the proposed method outperforms other ZSON methods for both seen and unseen target classes.

Active interaction with environments is one of the striking characteristics of robotic active vision, which allows robots to move to facilitate visual tasks. Recently, several embodied AI platforms have been proposed as the synthetic environments to study robotic active vision, without needing to interact in real world. However, by using synthetic data, model trained on these platforms will suffer performance degradation when applied in reality. In this letter, a real 3D embodied dataset is proposed for robotic active visual learning. The proposed dataset consists of real point cloud data densely collected in 7 real-world indoor scenes. In our embodied dataset, researchers are able to simulate the movements and interactions of robots in indoor environments and obtain real visual data, which will not lead to performance degradation in reality. Furthermore, we proposed a 3D divergency policy that can guide robots to move and collect data to improve visual performance in novel environments. The proposed policy is designed following a simple fact: a good 3D detector should produce consistent 3D detection results for the same object from different viewpoints. Therefore, our policy encourages the robot to explore the area where the detector generates different 3D bounding boxes for the same object and helps the robot improve its visual performance in novel scenes.

Intelligent manipulation of robots in an unstructured environment is an important application field of artificial intelligence, which means that robots must have the ability of autonomous cognition and decision-making. A typical example of this type of environment is a cluttered scene where objects are stacked and close together. In clutter, the target(s) may be one or more, and efficiently completing the target(s) grasping task is challenging. In this study, an efficient push-grasping method based on reinforcement learning is proposed for multiple target objects in clutter. The key point of this method is to consider the states of all the targets so that the pushing action can expand the grasping space of all targets as much as possible to achieve the minimum total number of pushing and grasping actions and then improve the efficiency of the whole system. At this point, we adopted the mask fusion of multiple targets, clearly defined the concept of graspable probability, and provided the reward mechanism of multi-target push-grasping. Experiments were conducted in both the simulation and real systems. The experimental results indicated that, compared with other methods, the proposed method performed better for multiple target objects and a single target in clutter. It is worth noting that our policy was only trained under simulation, which was then transferred to the real system without retraining or fine-tuning.