Robotic manipulation in cluttered environments requires synergistic planning among prehensile and non-prehensile actions. Previous works on sampling-based Task and Motion Planning (TAMP) algorithms, e.g. PDDLStream, provide a fast and generalizable solution for multi-modal manipulation. However, they are likely to fail in cluttered scenarios where no collision-free grasping approaches can be sampled without preliminary manipulations. To extend the ability of sampling-based algorithms, we integrate a vision-based Reinforcement Learning (RL) non-prehensile procedure, pusher . The pushing actions generated by pusher can eliminate interlocked situations and make the grasping problem solvable. Also, the sampling-based algorithm evaluates the pushing actions by providing rewards in the training process, thus the pusher can learn to avoid situations leading to irreversible failures. The proposed hybrid planning method is validated on a cluttered bin-picking problem and implemented in both simulation and real world. Results show that the pusher can effectively improve the success ratio of the previous sampling-based algorithm, while the sampling-based algorithm can help the pusher learn pushing skills.

As the industry shifts to automated manufacturing and the assembly of parts in smaller batches, there is a clear need for an efficient design of grippers. This paper presents a method for automated grasp planning and finger design for multiple parts using four grasp quality measures that capture the following important requirements for grasping: (i) uniform contact force distribution; (ii) better gravity wrench resistance; (iii) robustness against gripper positioning error; and (iv) ability to resist larger external wrench on the object. We introduce the fingertip score to quantify the grasp performance of a fingertip design over all the objects. The method takes the CAD model of the objects as the input and outputs the optimal grasp location and the best finger design. We use the method for a three-point grasp with a parallel jaw gripper. We validate our method on two sets of objects. Results show how each grasp quality measure behaves on different objects and the variation in the fingertip score with finger design. Finally, we test the effectiveness of the optimal finger design experimentally. The three-point grasp is suitable for grasping objects larger than is possible with shape-matching fingertips.

Task and motion planning (TAMP) for robotics manipulation necessitates long-horizon reasoning involving versatile actions and skills. While deterministic actions can be crafted by sampling or optimizing with certain constraints, planning actions with uncertainty, i.e., probabilistic actions, remains a challenge for TAMP. On the contrary, Reinforcement Learning (RL) excels in acquiring versatile, yet short-horizon, manipulation skills that are robust with uncertainties. In this letter, we design a method that integrates RL skills into TAMP pipelines. Besides the policy, a RL skill is defined with data-driven logical components that enable the skill to be deployed by symbolic planning. A plan refinement sub-routine is designed to further tackle the inevitable effect uncertainties. In the experiments, we compare our method with baseline hierarchical planning from both TAMP and RL fields and illustrate the strength of the method. The results show that by embedding RL skills, we extend the capability of TAMP to domains with probabilistic skills, and improve the planning efficiency compared to the previous methods.

Pruning is an essential agricultural practice for orchards. Proper pruning can promote healthier growth and optimize fruit production throughout the orchard’s lifespan. Robot manipulators have been developed as an automated solution for this repetitive task, which typically requires seasonal labor with specialized skills. While previous research has primarily focused on the challenges of perception, the complexities of manipulation are often overlooked. These challenges involve planning and control in both joint and Cartesian spaces to guide the end effector through intricate, obstructive branches. Our work addresses the behavior planning challenge for a robotic pruning system, which entails a multilevel planning problem in environments with complex collisions. In this article, we formulate the planning problem for a high-dimensional robotic arm in a pruning scenario, investigate the system’s intrinsic redundancies, and propose a comprehensive pruning workflow that integrates perception, modeling, and holistic planning. In our experiments, we demonstrate that more comprehensive planning methods can significantly enhance the performance of the robotic manipulator. Finally, we implement the proposed workflow on a real-world robot. As a result, this work complements previous efforts on robotic pruning and motivates future research and development in planning for pruning applications.

Real-time relative pose (RP) estimation is a cornerstone for effective multi-agent collaboration. When conventional global positioning infrastructure such as GPS is unavailable, the use of Ultra-Wideband (UWB) technology on each agent provides a practical means to measure inter-agent range. Due to UWB's precise range measurements and robust communication capabilities, external hardware installations are not needed. However, when only a single UWB device per agent is used, the relative pose between the agents can be unobservable, resulting in a complex solution space with multiple possible RPs. This paper proposes a novel method based on an Unscented Particle Filter (UPF) that fuses single UWB ranges with visual-inertial odometry (VIO). The proposed decentralized method solves the multi-modal solution in 3D (4-DoF) for the RP when it is unobservable. Moreover, a pseudo-state is introduced to correct the rotational drift of the agents. Through simulations and experiments involving two robots, the proposed solution was shown to be competitive and less computationally expensive than state-of-the-art algorithms. Additionally, the proposed solution provides all possible relative poses from the first measurement. The code and link to the video are available https://github.com/y2d2/UPF_RPE .