Slip detection is essential for robots to make robust grasping and fine manipulation. In this paper, a novel dynamic vision-based finger system for slip detection and suppression is proposed. We also present a baseline and feature based approach to detect object slips under illumination and vibration uncertainty. A threshold method is devised to autonomously sample noise and object feature events in real-time to improve slip detection and suppression. Moreover, a fuzzy based suppression strategy using incipient slip feedback is proposed for regulating the grip force. A comprehensive experimental study of our proposed approaches under uncertainty and system for high-performance precision manipulation are presented. We also propose a slip metric to evaluate such performance quantitatively. For a class of objects, results indicate that the system can effectively detect incipient slip events at a sampling rate of 2kHz (Δt = 500μs) and suppress them before a gross slip occurs. The event-based approach holds promises to high precision manipulation task requirement in industrial manufacturing and household services.

Robotic vision plays a key role for perceiving the environment in grasping applications. However, the conventional framed-based robotic vision, suffering from motion blur and low sampling rate, may not meet the automation needs of evolving industrial requirements. This paper, for the first time, proposes an event-based robotic grasping framework for multiple known and unknown objects in a cluttered scene. With advantages of microsecond-level sampling rate and no motion blur of event camera, the model-based and model-free approaches are developed for known and unknown objects’ grasping respectively. The event-based multi-view approach is used to localize the objects in the scene in the model-based approach, and then point cloud processing is utilized to cluster and register the objects. The proposed model-free approach, on the other hand, utilizes the developed event-based object segmentation, visual servoing and grasp planning to localize, align to, and grasp the targeting object. Using a UR10 robot with an eye-in-hand neuromorphic camera and a Barrett hand gripper, the proposed approaches are experimentally validated with objects of different sizes. Furthermore, it demonstrates robustness and a significant advantage over grasping with a traditional frame-based camera in low-light conditions.

This paper presents automated navigation control of an Unmanned Aerial Vehicle (UAV) based on visual data gathered by onboard camera. With depletion of easy resources and health, safety and environmental (HSE) challenges in exploiting newly found resources in hostile conditions are forcing oil and gas companies to look for robotic solutions for their problems. Pipelines carrying inflammable and toxic petroleum products are one of the most critical component of oil and gas industry. Being laid in extreme environmental conditions these critical assets require continuous inspection and maintenance. An UAV carrying non-contact sensors is proposed to use for external inspection of the visible pipeline sections. Till now UAVs are mostly used for general purpose visual surveillance while flying at very high altitude but in this novel work we propose accurate tracking and inspection while flying at very low altitude in close vicinity of the ground pipeline structure. Proposed automated inspection of the pipelines by the UAV has two stages firstly identification of the pipeline and secondly navigation control of the UAV for tracking of the pipeline structure. Both identification and tracking are based on visual data gathered by onboard cameras without any additional help of GPS information. This novel navigation control mechanism further has double layers of correction loops namely angular and lateral corrections. Navigation control uses heuristically tunes PID controllers and required geometrical parameters for angular and lateral corrections are extracted from visual data generated by onboard video camera.

In recent years, robotic sorting is widely used in the industry, which is driven by necessity and opportunity. In this paper, a novel neuromorphic vision-based tactile sensing approach for robotic sorting application is proposed. This approach has low latency and low power consumption when compared to conventional vision-based tactile sensing techniques. Two Machine Learning (ML) methods, namely, Support Vector Machine (SVM) and Dynamic Time Warping-K Nearest Neighbor (DTW-KNN), are developed to classify material hardness, object size, and grasping force. An Event-Based Object Grasping (EBOG) experimental setup is developed to acquire datasets, where 243 experiments are produced to train the proposed classifiers. Based on predictions of the classifiers, objects can be automatically sorted. If the prediction accuracy is below a certain threshold, the gripper re-adjusts and re-grasps until reaching a proper grasp. The proposed ML method achieves good prediction accuracy, which shows the effectiveness and the applicability of the proposed approach. The experimental results show that the developed SVM model outperforms the DTW-KNN model in term of accuracy and efficiency for real time contact-level classification.

This paper presents automated navigation control of an Unmanned Aerial Vehicle (UAV) for inspection of oil and gas pipelines. The pipelines carrying inflammable and toxic petroleum products require continuous inspection. An UAV equipped with non-contact sensors can be used for external inspection of the pipeline. Most of the UAVs used for surveillance purpose fly at very high altitude and same mechanism can not be used for low flying UAV while tracking ground structures such as pipelines. Current research work proposes novel two layers of autonomous navigation and tracking by UAV. First stage is identification of pipeline and second stage is tracking of it at low altitude. Identification and tracking of the pipelines as linear structure is based on image processing techniques without any assistance of GPS based localization. This autonomous navigation and tracking further has two layers of control loops, firstly angular correction and secondly lateral correction. Both the controllers are based on heuristically tuned simple PID controllers. Overall tracking has shown very good results in terms of stability and visibility of tracked structure from bottom camera of the UAV.

Event-based cameras are commonly leveraged to mitigate issues such as motion blur, low dynamic range, and limited time sampling, which plague conventional cameras. However, a lack of dedicated event-based datasets for benchmarking segmentation algorithms, especially those offering critical depth information for occluded scenes, has been observed. In response, this paper introduces a novel Event-based Segmentation Dataset (ESD), a high-quality event 3D spatial-temporal dataset designed for indoor object segmentation within cluttered environments. ESD encompasses 145 sequences featuring 14,166 manually annotated RGB frames, along with a substantial event count of 21.88 million and 20.80 million events from two stereo-configured event-based cameras. Notably, this densely annotated 3D spatial-temporal event-based segmentation benchmark for tabletop objects represents a pioneering initiative, providing event-wise depth, and annotated instance labels, in addition to corresponding RGBD frames. By releasing ESD, our aim is to offer the research community a challenging segmentation benchmark of exceptional quality.