This review paper focuses on vision and touch-based sensors known as visuotactile. The study of visuotactile sensation and perception became a multidisciplinary field of study by philosophers, psychologists, biologists, engineers, technologists, and roboticists in the fields of haptics, machine vision, and artificial intelligence and it dates back centuries. To the best of our knowledge, the earliest records of visuotactile sensor was not applied to robotics and was not even for hand or finger imprint analysis yet for recording the foot pressure distribution of a walking or standing human known as pedobarograph. Our review paper presents the different literature related to visuotactile sensors that lead to a high-resolution miniature pedobarograph-like sensor known as the GelSight sensor. Moreover, this review paper focuses on architecture, different techniques, hardware, and software development of GelSight sensor since 2009 with its applications in haptics, robotics, and computer vision.

In order to further maintain peace and provide security to people now a day, Closed-circuit television (CCTV) surveillance system is being utilized. This study focused on the design and implementation of a low cost smart security camera with night vision capability using Raspberry Pi (RPI) and OpenCV. The system was designed to be used inside a warehouse facility. It has human detection and smoke detection capability that can provide precaution to potential crimes and potential fire. The credit card size Raspberry Pi (RPI) with Open Source Computer Vision (OpenCV) software handles the image processing, control algorithms for the alarms and sends captured pictures to user's email via Wi-Fi. As part of its alarm system, it will play the recorded sounds: “intruder” or “smoke detected” when there is a detection. The system uses ordinary webcam but its IR filter was removed in order to have night vision capability.

This research proposes to automate the plate recognition process by installing an IP camera on a road and analyzing the video-feed to capture the vehicles along that road. The contours of the characters in a given plate image are detected, violated and isolated from the parent image. This results to segmented characters. Each of the characters are identified using a k nearest neighbors (kNN) algorithm. The kNN algorithm was trained using different sets of training data containing 36 characters each. The algorithm was tested on the previously segmented characters. The simulations show that an accuracy of 87.43% was achieved for the plate recognition algorithm using kNN at k = 1. Compared against existing character recognition techniques such as artificial neural networks (ANN), the difference in the accuracy is minimal. Moreover, the average processing time was 0.034 s.

This study describes the creation of a new type of compact skin-like silicone-based thermosensitive visuotactile sensor based on GelSight technology. The easy integration of this novel sensor into a complex visuotactile system capable of very rapid detection of temperature change (30°C/s) is unique in providing a system that parallels the withdrawal reflex of the human autonomic system to extreme heat. To the best of authors’ awareness, this is the first time a sensor that can trigger a sensory impulse like a withdrawal reflex of humans in robotics community. To attain this, we used thermochromic pigments color blue, orange, and black with a threshold of 31°C, 43°C, and 50°C, respectively on the gel material. Each pigment has the property of becoming translucent when its temperature threshold is reached, making it possible to stack thermochromic pigments of different colors and thresholds. The pigments were air-brushed on a low-cost commercially available transparent silicone sponge. We used MobileNetV2 and transfer learning to simulate tactile preprocessing in order to recognize five different objects. The new thermosensitive visuotactile sensor helped to achieve 97.3% tactile image classification accuracy of five different objects. Our novel thermosensitive visuotactile sensor could be of benefit in material texture analysis, telerobotics, space exploration, and medical applications.

Developing countries face the problem of crowded and congested roads because of inefficient implementation of traffic rules. Motorists ignore the rules because they are not apprehended and can get away easily. This paper proposes an intelligent traffic system that is able to automatically detect and apprehend traffic violators, specifically motorists who either swerve or block the pedestrian lane. The system is designed by integrating three processes: violation detection, plate localization and plate recognition. The violation detection and plate localization were realized using genetic algorithm while the plate recognition process was performed using an artificial neural network. The recognition of the plate number is highly dependent on the position of the detected vehicle with respect to the camera. Thus, the recognized plate number will only be supplementary information about the violator; the physical attributes of the vehicle which is captured by the violation detection process will be the main information on the violator. Based on the results of 48 images tested, the overall system was able to detect the mentioned violations and to identify the plate number of the vehicles that were detected as traffic violators, with an average accuracy of 90.67%, and program runtime of 1.34 seconds.

During open surgery, a surgeon relies not only on the detailed view of the organ being operated upon and on being able to feel the fine details of this organ but also heavily relies on the combination of these two senses. In laparoscopic surgery, haptic feedback provides surgeons information on interaction forces between instrument and tissue. There have been many studies to mimic the haptic feedback in laparoscopic-related telerobotics studies to date. However, cutaneous feedback is mostly restricted or limited in haptic feedback-based minimally invasive studies. We argue that fine-grained information is needed in laparoscopic surgeries to study the details of the instrument’s end and can convey via cutaneous feedback. We propose an exoskeleton haptic hand wearable which consists of five 4 × 4 miniaturized fingertip actuators, 80 in total, to convey cutaneous feedback. The wearable is described as modular, lightweight, Bluetooth, and WiFi-enabled, and has a maximum power consumption of 830 mW. Software is developed to demonstrate rapid tactile actuation of edges; this allows the user to feel the contours in cutaneous feedback. Moreover, to demonstrate the idea as an object displayed on a flat monitor, initial tests were carried out in 2D. In the second phase, the wearable exoskeleton glove is then further developed to feel 3D virtual objects by using a virtual reality (VR) headset demonstrated by a VR environment. Two-dimensional and 3D objects were tested by our novel untethered haptic hand wearable. Our results show that untethered humans understand actuation in cutaneous feedback just in a single tapping with 92.22% accuracy. Our wearable has an average latency of 46.5 ms, which is much less than the 600 ms tolerable delay acceptable by a surgeon in teleoperation. Therefore, we suggest our untethered hand wearable to enhance multimodal perception in minimally invasive surgeries to naturally feel the immediate environments of the instruments.