15 February 2023 in IEEE Robotics and Automation Letters
Marine debris is a problem both for the health of marine environments and for the human health since tiny pieces of plastic called “microplastics” resulting from the debris decomposition over the time are entering the food chain at any levels. For marine debris detection and removal, autonomous underwater vehicles (AUVs) are a potential solution. In this letter, we focus on the efficiency of AUV vision for real-time marine debris detection. First, we improved the efficiency of a class of state-of-the-art object detectors, namely EfficientDets [1], by 1.5% AP on D0, 2.6% AP on D1, 1.2% AP on D2 and 1.3% AP on D3 without increasing the GPU latency (see Fig. 1 ). Subsequently, we created and made publicly available a dataset for the detection of in-water plastic bags and bottles and trained our improved EfficientDets on this and on two public datasets for marine debris detection. Finally, we began the testing of real-time detection performance on a simulator of marine environments.
1 May 2022 in Field Robotics
26 April 2023 in IEEE Robotics and Automation Letters
Human-robot handover is a key capability of service robots, such as those used to perform routine logistical tasks for healthcare workers. Recent algorithms have achieved tremendous advances in object-agnostic end-to-end planar grasping with up to six degrees of freedom (DoF); however, compiling the requisite datasets is simply not feasible in many situations and many users consider the use of camera feeds invasive. This letter presents an end-to-end control system for the visual grasping of unseen objects with 6-DoF without infringing on the privacy or personal space of human counterparts. In experiments, the proposed Fed-HANet system trained using the federated learning framework achieved accuracy close to that of centralized non-privacy-preserving systems, while outperforming baseline methods that rely on fine-tuning. We also explores the use of a depth-only method and compares its performance to a state-of-the-art method, but ultimately emphasizes the importance of using RGB inputs for better grasp success. The practical applicability of the proposed system in a robotic system was assessed in a user study involving 12 participants. The dataset for training and all pretrained models are available at https://arg-nctu.github.io/projects/fed-hanet.html .
5 February 2024 in IEEE Robotics and Automation Letters
Robots are essential for tasks that are hazardous or beyond human capabilities. However, the results of the Defense Advanced Research Projects Agency (DARPA) Subterranean (SubT) Challenge revealed that despite various techniques for robot autonomy, human input is still required in some complex situations. Moreover, heterogeneous multirobot teams are often necessary. To manage these teams, effective user interfaces to support humans are required. Accordingly, we present a framework that enables intuitive oversight of a robot team through immersive virtual reality (VR) visualizations. The framework simplifies the management of complex navigation among movable obstacles (NAMO) tasks, such as search-and-rescue tasks. Specifically, the framework integrates a simulation of the environment with robot sensor data in VR to facilitate operator navigation, enhance robot positioning, and greatly improve operator situational awareness. The framework can also boost mission efficiency by seamlessly incorporating autonomous navigation algorithms, including NAMO algorithms, to reduce detours and operator workload. The framework is effective for operating in both simulated and real scenarios and is thus ideal for training or evaluating autonomous navigation algorithms. To validate the framework, we conducted user studies ( $N = 53$ ) on the basis of the DARPA SubT Challenge's search-and-rescue missions.
8 February 2024 in Solar Energy Materials and Solar Cells