Collecting in shallow water (water depth: ~30 m) is an emerging field that requires robotics for replacing human divers. Soft robots have several promising features (e.g., safe interaction with the environments, lightweight, etc.) for performing such tasks. In this article, we developed an underwater robotic system with a three-degree-of-freedom (3-DoF) soft manipulator for spatial delicate grasping in shallow water. First, we present the design and fabrication of the soft manipulator with an opposite-bending-and-stretching structure (OBSS). Then, we proposed a simple and efficient kinematics method for controlling the spatial location and trajectory of the soft manipulator’s end effector. The inverse kinematics of the OBSS manipulator can be solved efficiently (computation time: 8.2 ms). According to this inverse kinematics method, we demonstrated that the OBSS soft manipulator could track complex two-dimensional and three-dimensional trajectories, including star, helix, etc. Further, we performed real-time closed-loop pick-and-place experiments of the manipulator with binocular and on-hand cameras in a lab aquarium. Hydrodynamic experiments showed that the OBSS soft manipulator produced little force (less than 0.459 N) and torque (less than 0.228 N·m), which suggested its low-inertia feature during the underwater operation. Finally, we demonstrated that the underwater robotic system with the OBSS soft manipulator successfully collected seafood animals at the bottom of the natural oceanic environment. The robot successfully collected eight sea echini and one sea cucumber within 20 minutes at a water depth of around 10 m.

Low visual quality has prevented underwater robotic vision from a wide range of applications. Although several algorithms have been developed, real time and adaptive methods are deficient for real-world tasks. In this paper, we address this difficulty based on generative adversarial networks (GAN), and propose a GAN-based restoration scheme (GAN-RS). In particular, we develop a multibranch discriminator including an adversarial branch and a critic branch for the purpose of simultaneously preserving image content and removing underwater noise. In addition to adversarial learning, a novel dark channel prior loss also promotes the generator to produce realistic vision. More specifically, an underwater index is investigated to describe underwater properties, and a loss function based on the underwater index is designed to train the critic branch for underwater noise suppression. Through extensive comparisons on visual quality and feature restoration, we confirm the superiority of the proposed approach. Consequently, the GAN-RS can adaptively improve underwater visual quality in real time and induce an overall superior restoration performance. Finally, a real-world experiment is conducted on the seabed for grasping marine products, and the results are quite promising. The source code is publicly available ¹¹ [Online]. Available: https://github.com/SeanChenxy/GAN_RS..

In this article, a robot system for intelligent water surface cleaner named IWSCR is developed to collect floating plastic garbage. It is able to accomplish three major tasks autonomously, i.e., cruise and detection, tracking and steering, and grasping and collection. The challenges behind these tasks involve how to realize the accurate and real-time garbage detection, how to resist the disturbances while IWSCR conducts vision-based steering, and how to grasp the floating garbage reliably despite the turbulent conditions on the surface of the water. To overcome these difficulties, three key techniques are proposed for IWSCR. First, the YOLOv3 network, which is widely applied in the high speed and accuracy object detection field, is trained on the proposed floating garbage dataset to realize accurate and real-time garbage detection. Next, to improve the ability of resisting disturbances, a control law based on the sliding-mode controller is proposed for vision-based steering. Furthermore, inspired by the stability of floating bottles in fluid, a feasible grasping strategy is utilized for IWSCR. Finally, the experimental results demonstrate that IWSCR is competent to carry out the task of water surface cleaning.

In this article, an extended state observer (ESO)-based controller with a model predictive governor is designed for 3-D trajectory tracking of underactuated underwater vehicles. The proposed control scheme takes three primary challenges including underactuated property, velocity constraint, and lumped disturbance into consideration. With respect to the model predictive governor, an underactuated kinematic tracking error model is utilized to produce reference velocities. Meanwhile, a heading angle compensation mechanism is utilized to avoid the steady tracking errors resulting from dynamics coupling of the vehicle. Besides, an ESO is designed to estimate the lumped disturbances and unmeasured velocity states. Based on the ESO, a kinetic controller is offered to accomplish the precise velocity tracking only in virtue of the position and orientation information. Note that this article details both the design process of the control scheme and rigorous theoretical analysis. Eventually, simulation and experimental results demonstrate the feasibility and superiority of the proposed method. Notably, this work lays the foundation for the underactuated trajectory tracking control in complicated and turbulent underwater environments.

Object detection has been vigorously investigated for years but fast accurate detection for real-world scenes remains a very challenging problem. Overcoming drawbacks of single-stage detectors, we take aim at precisely detecting objects for static and temporal scenes in real time. Firstly, as a dual refinement mechanism, a novel anchor-offset detection is designed, which includes an anchor refinement, a feature location refinement, and a deformable detection head. This new detection mode is able to simultaneously perform two-step regression and capture accurate object features. Based on the anchor-offset detection, a dual refinement network (DRNet) is developed for high-performance static detection, where a multi-deformable head is further designed to leverage contextual information for describing objects. As for temporal detection in videos, temporal refinement networks (TRNet) and temporal dual refinement networks (TDRNet) are developed by propagating the refinement information across time. We also propose a soft refinement strategy to temporally match object motion with the previous refinement. Our proposed methods are evaluated on PASCAL VOC, COCO, and ImageNet VID datasets. Extensive comparisons on static and temporal detection verify the superiority of DRNet, TRNet, and TDRNet. Consequently, our developed approaches run in a fairly fast speed, and in the meantime achieve a significantly enhanced detection accuracy, i.e., 84.4% mAP on VOC 2007, 83.6% mAP on VOC 2012, 69.4% mAP on VID 2017, and 42.4% AP on COCO. Ultimately, producing encouraging results, our methods are applied to online underwater object detection and grasping with an autonomous system. Codes are publicly available at https://github.com/SeanChenxy/TDRN.

To tackle the problem of aquatic environment pollution, a vision-based autonomous underwater garbage cleaning robot has been developed in our laboratory. W