Adding haptic feedback has been reported to improve the outcome of minimally invasive robotic surgery. In this study, we seek to determine whether an algorithm based on simulating responses of a cutaneous afferent population can be implemented to improve the performance of presenting haptic feedback for robot-assisted surgery. We propose a bio-inspired controlling model to present vibration and force feedback to help surgeons localize underlying structures in phantom tissue. A single pair of actuators was controlled by outputs of a model of a population of cutaneous afferents based on the pressure signal from a single sensor embedded in surgical forceps. We recruited 25 subjects including 10 expert surgeons to evaluate the performance of the bio-inspired controlling model in an artificial palpation task using the da Vinci surgical robot. Among the control methods tested, the bio-inspired system was unique in allowing both novices and experts to easily identify the locations of all classes of tumors and did so with reduced contact force and tumor contact time. This work demonstrates the utility of our bio-inspired multi-modal feedback system, which resulted in superior performance for both novice and professional users, in comparison to a traditional linear and the existing piecewise discrete algorithms of haptic feedback.

Information transmission is a fundamental issue for human-computer interaction (HCI). Traditional interaction methods through visual, voice, and force haptics are very mature. However, the thermal perception (TP) for HCI is not studied in depth. This work proposes the TP-based information transmission framework. Firstly, we investigated the human hand-object contact heat transfer model and the temperature perception resolution of the hand and verified the feasibility of spatiotemporal temperature stimulation for information transmission by simulations. Then, a thermal device was designed, which utilized a 7×5 Peltier array, a water cooler, temperature sensors, and a control module to realize various static and dynamic spatiotemporal temperature patterns stimulation. Finally, we implemented a device prototype and recruited 20 subjects for experimental studies. The results show that the device can display various temperature patterns and provide thermal stimulations with high precision and speed. Furthermore, the subjects can accurately recognize different temperature values, icons, codes, and waveforms with their palm and fingers after a few times of training, which validates the TP-based information transmission method. Therefore, people can apply this method to interact with machines for information feedback, virtual reality, augmented reality, etc.

Vibration acceleration signals allow humans to perceive the surface characteristics of textures during tool-surface interactions. However, acquiring acceleration signals requires a specialized system, which is relatively expensive. Conversely, visual images are more accessible than acceleration signals, and generative models can convert visual images into vibration acceleration signals. Utilizing generative models to generate vibration acceleration signals from visual data circumvents the need for time-consuming actual measurements. Furthermore, this approach can be applied to robot-related tasks. This letter presents a cross-modal adversarial variational autoencoder (CM-AVAE) for visual-to-tactile data generation. Our model incorporates latent space learning from variational autoencoders (VAEs) into generative adversarial networks (GANs) and maps the generator's decoder feature vectors to the discriminator. In addition, a public dataset is chosen to train the model, and relevant evaluation metrics are established to evaluate the model's generated results. The results generated by the CM-AVAE model show significant improvement in objective experiments compared to the baseline models. Furthermore, subjective experimental outcomes also surpass those of the baseline models. Ablation study shows that CM-AVAE introduces latent space learning and maps the decoder feature vectors in the generator to the discriminator, significantly improving the quality of cross-modal data generation.

Tactile information about an object can only be extracted from population responses of tactile receptors and their afferents. Thus, to best control tactile information in robots, neuroprostheses or haptic devices, inputs should represent responses from full populations of afferents. Here, we describe a simplified model that recreates afferent population responses of thousands of tactile afferents in a personal computer. The whole model includes a resistance network model to simplify the skin mechanics and an improved version of a single unit model that we have previously described. The whole model was implemented by short and efficient python code. The parameters of the model were fit based on a simple vibrating stimulus, but the simulated outputs generalize to match receptive field sizes, edge enhancement, and neurophysiological responses to dot textures, embossed letters and curved surfaces. We discuss how to use this work to model haptic perception and provide guidance in designing and controlling highly realistic tactile interfaces in robots, neural prostheses and haptic devices.

Haptic rendering of compliance is widely used in human–computer haptic interaction. Haptic impressions of virtual objects are usually controlled through rendering algorithms and devices. However, subjective feelings of compliance are easily affected by physical properties of objects, interactive modes, and so on. So it is important to ascertain the mapping relations between controlled physical parameters and subjective perceptual feelings. In this paper, a multi-layered perceptual model was built based on psychophysical experiments to discuss these relationships in a simplified scene. Interactive signals of physical stimuli are collected by the physical receptor layer, handled by the subjective classifier layer and finally generate the evaluation results of compliance. The physical perceptual layer is used to extract useful interaction features affecting perceptual results. The subjective classifier layer is used to analyze the perceptual dimensionality of the compliance perception. The final aim of the model is to determine the mapping relationships between interaction features and dimensions of perception space. Interactive features are extracted from the interaction data collected during the exploring process, perceptual dimensionality of the compliance perception was analyzed by the factor analysis method, and relations between hierarchical layers were obtained by multi-linear regression analysis. A verification test was performed to show whether the proposed model can predict the perceptual result of new samples well. The results indicate that the model was reliable to estimate the perceptual results of compliance with an accuracy of approximately 90%. This paper may contribute a lot to the design and improvement of human-computer interaction and intelligent sensing system.

This work presents a pieces of Python code to rapidly simulate the spiking responses of large numbers of single cutaneous tactile afferents with millisecond precision. To simulate the spike responses of all the major types of cutaneous tactile afferents, we proposed an electromechanical circuit model, in which a two-channel filter was developed to characterize the mechanical selectivity of tactile receptors, and a spike synthesizer was designed to recreate the action potentials evoked in afferents. The parameters of this model were fitted using previous neurophysiological datasets. Several simulation examples were presented in this paper to reproduce action potentials, sensory adaptation, frequency characteristics and spiking timing for each afferent type. The results indicated that the simulated responses matched previous neurophysiological recordings well. The model allows for a real-time reproduction of the spiking responses of about 4,000 tactile units with a timing precision of <6 ms. The current work provides a valuable guidance to designing highly realistic tactile interfaces such as neuroprosthesis and haptic devices