In our daily life, we, human beings use our hands in various ways for most of our day-to-day activities. Tracking the position, orientation and articulation of human hands has a variety of applications including gesture recognition, robotics, medicine and health care, design and manufacturing, art and entertainment across multiple domains. Out of the various tracking methods, vision based tracking is an efficient and widely used method. Several devices have been developed by researchers and engineers to track objects using vision. The Leap Motion controller is one such device. However, visual tracking is an equally complex and challenging task due to several factors like higher dimensional data from hand motion, higher speed of operation, self-occlusion, etc. This paper puts forth a novel method for tracking the finger tips of human hand using two distinct sensors and combining their data by sensor fusion technique. The proposed method is tested using standard human hand gestures and the results are discussed. Finally, a soft robotic gripper was operated remotely based on Leap Motion hand tracking and the proposed sensor fusion method.

Origami-based flexible, compliant, and bio-inspired robots are believed to permit a range of medical applications within confined environments. In this article, we experimentally demonstrated an origami-inspired deployable surgical retractor with the controllable stiffness mechanism that can facilitate safer instrument–tissue interaction in comparison to their rigid counterparts. When controllable negative-pressure is applied to the jammed origami retractor module, it becomes more rigid, increasing its strength. To quantify origami-modules strength further, we demonstrated performances of retractor based on the Daler–Rowney Canford paper (38 grams per square meter (gsm)) and sandpaper of 1000 grit. Experiments on the proposed retractor prototype elucidated sandpaper-based retractor can outperform paper-38-gsm retractor for facelift incision with the width of more than 9 cm. Though 38 gsm Canford paper comprised of thin layers, 16 times lesser in thickness than sandpaper, experiments proved its comparable layer jamming (LJ) performance. We leverage the advantage of the LJ mechanism to tune retractor stiffness, allowing the instrument to hold and separate a facelift incision to mitigate the likelihood of surgical complications. The retractor is equipped with a custom-made printed conductive ink-based fabric piezoresistive tactile sensor to assist clinicians with tissue-retractor interaction force information. The proposed sensor showed a linear relationship with the applied force and has a sensitivity of 0.833 N−1. Finally, cadaver experiments exhibit an effective origami-inspired surgical retractor for assisting surgeons and clinicians in the near future.

Intra-abdominal pressure (IAP) is closely correlated with intra-abdominal hypertension (IAH) and abdominal compartment syndrome (ACS) diagnoses, indicating the need for continuous monitoring. Early intervention for IAH and ACS has been proven to reduce the rate of morbidity. However, the current IAP monitoring method is a tedious process with a long calibration time for a single time point measurement. Thus, there is the need for an efficient and continuous way of measuring IAP. Herein, a stretchable capacitive pressure sensor with controlled microstructures embedded into a cylindrical elastomeric mold, fabricated as a pressure sensing sleeve, is presented. The sensing sleeve can be readily deployed onto intrabody catheter balloons for pressure measurement at the site. The thin and highly conformable nature of the pressure sensing sleeve captures the pressure change without hindering the functionality of the foley catheter balloon.

The physical interaction of robots with its environment requires a touch sensory system to function effectively. This sensory system measures the robot-environment interactive force and pressure information that can be converted to electrical signals. We fabricated and compared a series of of electronic textile (e-textile) tactile sensors using flexible piezoresistive materials and two types of conductive textile materials with varying layer constructions. Compared with conventional rigid counterparts, the tactile sensors have advantages in terms of light weight, flexible simple design, high sensitivity, low power consumption, and stretchability and thus is operable at varying curvy surfaces and dynamic forces.

A number of surgical tools are widely being used by clinicians and dentists for various purposes from simple exploration to complex oral surgeries. Most of such medical devices rely on direct vision or oral confirmation from the patients as a means of feedback to guide the clinician throughout the procedure. However, a tactile sense of palpation or device-tissue interactions as often required by clinicians would greatly assist in improving the control and safety with better precision thus increasing the efficiency of the whole practice. Tactile sensing is an emerging field in surgical robotics focused mainly on studying the interactions between the confined tissue environments and the instruments based on contact information. This paper explains the design, fabrication, and working of one such soft tactile sensor made using a flexible piezoresistive material and conductive ink, and then wrapped with a biocompatible hydrogel polymer matrix for safer human-tissue interactions. Successful cadaver experiments have shown that the system developed herein will be of effective in providing tactile sensing assistance to clinicians and dentists in the near future.

Bimanual telemanipulation is vital for facilitating robots to complete complex and dexterous tasks that involve two handheld objects under teleoperation scenarios. However, the bimanual configuration introduces higher complexity, dynamics, and uncertainty, especially in those uncontrolled and unstructured environments, which require more advanced system integration. This paper presents a bimanual robotic teleoperation architecture with modular anthropomorphic hybrid grippers for the purpose of improving the telemanipulation capability under unstructured environments. Generally, there are two teleoperated subsystems within this architecture. The first one is the Leap Motion Controller and the anthropomorphic hybrid robotic grippers. Two 3D printed anthropomorphic hybrid robotic grippers with modular joints and soft layer augmentations are designed, fabricated, and equipped for telemanipulation tasks. A Leap Motion Controller is used to track the motion of two human hands, while each hand is utilized to teleoperate one robotic gripper. The second one is the haptic devices and the robotic arms. Two haptic devices are adopted as the master devices while each of them takes responsibility for one arm control. Based on such a framework, an average RMSE (root-mean-square-error) value of 0.0204 rad is obtained in joint tracking. Nine sign-language demonstrations and twelve object grasping tasks were conducted with the robotic gripper teleoperation. A challenging bimanual manipulation task for an object with 5.2 kg was well addressed using the integrated teleoperation system. Experimental results show that the proposed bimanual teleoperation system can effectively handle typical manipulation tasks, with excellent adaptabilities for a wide range of shapes, sizes, and weights, as well as grasping modes.