The impressive results of applying deep neural networks in tasks such as object recognition, language translation, and solving digital games are largely attributed to the availability of massive amounts of high quality labelled data. However, despite numerous promising steps in incorporating these techniques in robotic tasks, the cost of gathering data with a real robot has halted the proliferation of deep learning in robotics. In this work, a plugin for the Gazebo simulator is presented, which allows rapid generation of synthetic data. By introducing variations in simulator-specific or irrelevant aspects of a task, one can train a model which exhibits some degree of robustness against those aspects, and ultimately narrow the reality gap between simulated and real-world data. To show a use-case of the developed software, we build a new dataset for detection and localisation of three object classes: box, cylinder and sphere. Our results in the object detection and localisation task demonstrate that with small datasets generated only in simulation, one can achieve comparable performance to that achieved when training on real-world images.
This work deals with the problem of detecting cylindrical shapes, commonly found in household and industrial environments, using 3D point cloud information from consumer RBG-D cameras. Existing approaches are fragile in the presence of clutter, in particular flat surfaces, leading to errors during the orientation estimation process that compromise the whole method. We address the aforementioned problem with a novel soft voting scheme that incorporates curvature information in the orientation voting phase. For each potential cylinder point, the principal curvature direction is combined with the normal vector to disambiguate candidate orientations. A set of experiments with synthetically generated data are used to assess the robustness of our method with different levels of clutter and noise. The results demonstrate that incorporating the principal curvature direction within the orientation voting process allows for large improvements on cylinders' parameters estimation. Qualitative results with point clouds acquired from consumer RGB-D cameras, confirm the advantages of the proposed approach.
Large vessels such as container ships rely on experienced pilots with extensive knowledge of the local streams and tides responsible for maneuvering the vessel to its desired location. This work proposes estimating time-to-collision (TTC) between moving objects (i.e., vessels) using real-time video data captured from aerial drones in dynamic maritime environments. Our deep-learning-based methods utilize features optimized with realistic virtually generated data for reliable and robust object detection, segmentation, and tracking. Furthermore, we use rotated bounding box representations, obtained from fine semantic segmentation of objects, for enhanced TTC estimation accuracy. We intuitively present collision estimates as collision arrows that gradually change color to red to indicate an imminent collision. Experiments conducted in a realistic dockyard virtual environment show that our approaches precisely, robustly, and efficiently predict TTC between dynamic objects seen from a top-view, with a mean error and a standard deviation of 0.358 and 0.114 s, respectively, in a worst-case scenario.