This paper studies the tracking control problem for an uncertain ${n}$ -link robot with full-state constraints. The rigid robotic manipulator is described as a multiinput and multioutput system. Adaptive neural network (NN) control for the robotic system with full-state constraints is designed. In the control design, the adaptive NNs are adopted to handle system uncertainties and disturbances. The Moore-Penrose inverse term is employed in order to prevent the violation of the full-state constraints. A barrier Lyapunov function is used to guarantee the uniform ultimate boundedness of the closed-loop system. The control performance of the closed-loop system is guaranteed by appropriately choosing the design parameters. Simulation studies are performed to illustrate the effectiveness of the proposed control.

Grasping unknown objects in unstructured environments is one of the most challenging and demanding tasks for robotic bin picking systems. Developing a holistic approach is crucial to building such dexterous bin picking systems to meet practical requirements on speed, cost and reliability. Proposed datasets so far focus only on challenging sub-problems and are therefore limited in their ability to leverage the complementary relationship between individual tasks. In this paper, we tackle this holistic data challenge and design MetaGraspNetV2, an all-in-one bin picking dataset consisting of (i) a photo-realistic dataset with over 296k images, which has been created through physics-based metaverse synthesis; and (ii) a real-world test dataset with 3.2k images featuring task-specific difficulty levels. Both datasets provide full annotations for amodal panoptic segmentation, object relationship detection, occlusion reasoning, 6-DoF pose estimation, and grasp detection for a parallel-jaw as well as a vacuum gripper. Extensive experiments demonstrate that our dataset outperforms state-of-the-art datasets in object detection, instance segmentation, amodal detection, parallel-jaw grasping, and vacuum grasping. Furthermore, leveraging the potential of our data for building holistic perception systems, we propose a single-shot-multi-pick (SSMP) grasping policy for scene understanding accelerated fast picking in high clutter. SSMP reasons about suitable manipulation orders for blindly picking multiple items given a single image acquisition. Physical robot experiments demonstrate that SSMP effectively speeds up cycle times through reducing image acquisitions by more than 47% while providing better grasp performance compared to state-of-the-art bin picking methods. Note to Practitioners —In robotic bin picking, most proposed methods and datasets focus on solving only one aspect of the grasping task, such as grasp point detection, object detection, or relationship reasoning. They do not address practical aspects such as the widespread use of vacuum grasp technology or the need for short cycle times. In practice, however, efficient bin picking solutions often rely on multiple task-specific methods. Hence, having one dataset for a large variety of vision-related tasks in robotic picking reduces data redundancy and enables the development of holistic methods. While deep learning has been proven highly effective for bin picking vision systems, it demands large, high-quality training datasets. Collecting such datasets in the real-world, while assuring label quality and consistency, is prohibitively expensive and time-consuming. To overcome these challenges, we set up a photo-realistic metaverse data generation pipeline and create a large-scale synthetic training dataset. Furthermore, we design a comprehensive real-world dataset for testing. Unlike previously proposed datasets, our datasets provide difficulty levels and annotations in simulation and real-world for a comprehensive list of high-level tasks, including amodal object detection, scene layout reasoning, and grasp detection. In real-world applications, cycle time is a critical factor affecting the productivity and profitability of a robotic system. We tackle time-efficiency through scene understanding and demonstrate the capability of our data regarding holistic system development by proposing a single-shot-multi-pick (SSMP) policy. Our SSMP algorithm, trained exclusively on our synthetic data, distinguishes between uncovered and occluded items, and infers specific manipulation orders to perform multiple blind picks in a single shot. Physical robot experiments show that SSMP was able to reduce image acquisitions by more than 47% without compromising grasp performance. This clearly demonstrates that SSMP, together with our dataset, paves the way for application-oriented research in time-critical bin picking.

As technology advances, Human-Robot Interaction (HRI) is boosting overall system efficiency and productivity. However, allowing robots to be present closely with humans will inevitably put higher demands on precise human motion tracking and prediction. Datasets that contain both humans and robots operating in the shared space are receiving growing attention as they may facilitate a variety of robotics and human-systems research. Datasets that track HRI with rich information other than video images during daily activities are rarely seen. In this paper, we introduce a novel dataset that focuses on social navigation between humans and robots in a future-oriented Wholesale and Retail Trade (WRT) environment (https://uf-retail-cobot-dataset.github.io/). Eight participants performed the tasks that are commonly undertaken by consumers and retail workers. More than 260 minutes of data were collected, including robot and human trajectories, human full-body motion capture, eye gaze directions, and other contextual information. Comprehensive descriptions of each category of data stream, as well as potential use cases are included. Furthermore, analysis with multiple data sources and future directions are discussed.

A rapid rise in the recycling and remanufacturing of end-of-use electronic waste (e-waste) has been observed due to multiple factors including our increased dependence on electronic products and the lack of resources to meet the demand. E-waste disassembly, which is the operation of extracting valuable components for recycling purposes, has received ever increasing attention as it can serve both the economy and the environment. Traditionally, e-waste disassembly is labor intensive with significant occupational hazards. To reduce labor costs and enhance working efficiency, collaborative robots (cobots) might be a viable option and the feasibility of deploying cobots in high-risk or low value-added e-waste disassembly operations is of tremendous significance to be investigated. Therefore, the major objective of this study was to evaluate the effects of working with a cobot during e-waste disassembly processes on human workload and ergonomics through a human subject experiment. Statistical results revealed that using a cobot to assist participants with the desktop disassembly task reduced the sum of the NASA-TLX scores significantly compared to disassembling by themselves (p = 0.001). With regard to ergonomics, a significant reduction was observed in participants’ mean L5/S1 flexion angle as well as mean shoulder flexion angle on both sides when working with the cobot (p < 0.001). However, participants took a significantly longer time to accomplish the disassembly task when working with the cobot (p < 0.001), indicating a trade-off of deploying cobot in the e-waste disassembly process. Results from this study could advance the knowledge of how human workers would behave and react during human-robot collaborative e-waste disassembly tasks and shed light on the design of better HRC for this specific context.