Learning from demonstration (LfD) has been used to help robots to implement manipulation tasks autonomously, in particular, to learn manipulation behaviors from observing the motion executed by human demonstrators. This paper reviews recent research and development in the field of LfD. The main focus is placed on how to demonstrate the example behaviors to the robot in assembly operations, and how to extract the manipulation features for robot learning and generating imitative behaviors. Diverse metrics are analyzed to evaluate the performance of robot imitation learning. Specifically, the application of LfD in robotic assembly is a focal point in this paper.
In recent years, with the continuous development of autonomous driving technology, 3D object detection has naturally become a key focus in the research of perception systems for autonomous driving. As the most crucial component of these systems, 3D object detection has gained significant attention. Researchers increasingly favor the deep learning framework Transformer due to its powerful long-term modeling ability and excellent feature fusion advantages. A large number of excellent Transformer-based 3D object detection methods have emerged. This article divides the methods based on data sources. Firstly, we analyze different input data sources and list standard datasets and evaluation metrics. Secondly, we introduce methods based on different input data and summarize the performance of some methods on different datasets. Finally, we summarize the limitations of current research, discuss future directions and provide some innovative perspectives.
21 October 2016 in IEEE Transactions on Cognitive and Developmental Systems
Robotic hand-eye coordination is recognized as an important skill to deal with complex real environments. Conventional robotic hand-eye coordination methods merely transfer stimulus signals from robotic visual space to hand actuator space. This paper introduces a reverse method. Build another channel that transfers stimulus signals from robotic hand space to visual space. Based on the reverse channel, a human-like behavior pattern: “Stop-to-Fixate,” is imparted to the robot, thereby giving the robot an enhanced reaching ability. A visual processing system inspired by the human retina structure is used to compress visual information so as to reduce the robot's learning complexity. In addition, two constructive neural networks establish the two sensory delivery channels. The experimental results demonstrate that the robotic system gradually obtains a reaching ability. In particular, when the robotic hand touches an unseen object, the reverse channel successfully drives the visual system to notice the unseen object.
The paper presents an approach to applying a classifier ensemble to identify human body gestures, so as to control a robot to write Chinese characters. Robotic handwriting ability requires complicated robotic control algorithms. In particular, the Chinese handwriting needs to consider the relative positions of a character's strokes. This approach derives the font information from human gestures by using a motion sensing input device. Five elementary strokes are used to form Chinese characters, and each elementary stroke is assigned to a type of human gestures. Then, a classifier ensemble is applied to identify each gesture so as to recognize the characters that gestured by the human demonstrator. The classier ensemble's size is reduced by feature selection techniques and harmony search algorithm, thereby achieving higher accuracy and smaller ensemble size. The inverse kinematics algorithm converts each stroke's trajectory to the robot's motor values that are executed by a robotic arm to draw the entire character. Experimental analysis shows that the proposed approach can allow a human to naturally and conveniently control the robot in order to write many Chinese characters.
Agricultural robots offer a viable solution to the critical challenges of productivity and sustainability of modern agriculture. The widespread deployment of agricultural robotic fleets, however, is still hindered by the overall system's complexity, requiring the integration of several nontrivial components for the operation of each robot but also the orchestration of robots working with each other and human workers. This paper proposes a topological map as the unifying representation and computational model to facilitate the smooth deployment of robotic fleets in agriculture. This topological abstraction of the system state (probabilistic estimates of the functional states and locations of the agents in the environment abstracted to the topological map representations—e.g., human picker at a node and robot traversing on edge ) results in an efficient representation of large‐scale environments (in many hectares), but also offers the scalable and efficient operation of the entire fleet and allows for ex situ modeling and analysis of operations. The practical use of the proposed framework is demonstrated in a horticultural use case with a fleet of robots supporting the work of human fruit pickers. The critical components of the system are analyzed and evaluated in deployment in both realistic digital twin and real‐life soft fruit farms of different scales, demonstrating the scalability and effectiveness of the proposed framework. The presented framework is general and should be easy to adopt in other multirobot/multihuman scenarios, such as warehouse logistics, cleaning, and maintenance of public spaces.
This paper presents a novel approach for a robot to conduct assembly tasks, namely robot learning from human demonstrations. The learning of robotic assembly task is divided into two phases: teaching and reproduction. During the teaching phase, a wrist camera is used to scan the object on the workbench and extract its SIFT feature. The human demonstrator teaches the robot to grasp the object from the effective position and orientation. During the reproduction phase, the robot uses the learned knowledge to reproduce the grasping manipulation autonomously. The robustness of the robotic assembly system is evaluated through a series of grasping trials. The dual-arm Baxter robot is used to perform the Peg-in-Hole task by using the proposed approach. Experimental results show that the robot is able to accomplish assembly task by learning from human demonstration without traditional dedicated programming.