Background: In this work, we present a novel sensory substitution system that enables to learn three dimensional digital information via touch when vision is unavailable. The system is based on a mouse-shaped device, designed to jointly perceive, with one finger only, local tactile height and inclination cues of arbitrary scalar fields. The device hosts a tactile actuator with three degrees of freedom: elevation, roll and pitch. The actuator approximates the tactile interaction with a plane tangential to the contact point between the finger and the field. Spatial information can therefore be mentally constructed by integrating local and global tactile cues: the actuator provides local cues, whereas proprioception associated with the mouse motion provides the global cues.Methods: The efficacy of the system is measured by a virtual/real object-matching task. Twenty-four gender and age-matched participants (one blind and one blindfolded sighted group) matched a tactile dictionary of virtual objects with their 3D-printed solid version. The exploration of the virtual objects happened in three conditions, i.e., with isolated or combined height and inclination cues. We investigated the performance and the mental cost of approximating virtual objects in these tactile conditions.Results: In both groups, elevation and inclination cues were sufficient to recognize the tactile dictionary, but their combination worked at best. The presence of elevation decreased a subjective estimate of mental effort. Interestingly, only visually impaired participants were aware of their performance and were able to predict it.Conclusions: The proposed technology could facilitate the learning of science, engineering and mathematics in absence of vision, being also an industrial low-cost solution to make graphical user interfaces accessible for people with vision loss.

Aberrant motor-sensory predictive functions have been linked to symptoms of psychosis, particularly reduced attenuation of self-generated sensations and misattribution of self-generated actions. Building on the parallels between prediction of self- and other-generated actions, this study aims to investigate whether individuals with psychosis also demonstrate abnormal perceptions and predictions of others’ actions. Patients with psychosis and matched controls completed a two-alternative object size discrimination task. In each trial, they observed reaching actions towards a small and a large object, with varying levels of temporal occlusion ranging from 10% to 80% of movement duration. Their task was to predict the size of the object that would be grasped. We employed a novel analytic approach to examine how object size information was encoded and read out across progressive levels of occlusion with single-trial resolution. Patients with psychosis exhibited an overall pattern of reduced and discontinuous evidence integration relative to controls, characterized by a period of null integration up to 20% of movement duration, during which they did not read any size information. Surprisingly, this drop in accuracy in the initial integration period was not accompanied by a reduction in confidence. Difficulties in action prediction were correlated with the severity of negative symptoms and impaired functioning in social relationships.

The advent of mixed reality (MR) systems has revolutionized human–computer interactions by seamlessly integrating virtual elements with the real world. Devices like the HoloLens 2 (HL2) enable intuitive, hands-free interactions through advanced hand-tracking technology, making them valuable in fields such as education, healthcare, engineering, and training simulations. However, despite the growing adoption of MR, there is a noticeable lack of comprehensive comparisons between the hand-tracking accuracy of the HL2 and high-precision benchmarks like motion capture systems. Such evaluations are essential to assess the reliability of MR interactions, identify potential tracking limitations, and improve the overall precision of hand-based input in immersive applications. This study aims to assess the accuracy of HL2 in tracking hand position and measuring kinematic hand parameters, including joint angles and lateral pinch span (distance between thumb and index fingertips), using its tracking data. To achieve this, the Vicon motion capture system (VM) was used as a gold-standard reference. Three tasks were designed: (1) finger tracing of a 2D pattern in 3D space, (2) grasping various common objects, and (3) lateral pinching of objects with varying sizes. Task 1 tests fingertip tracking, Task 2 evaluates joint angle accuracy, and Task 3 examines the accuracy of pinch span measurement. In all tasks, HL2 and VM simultaneously recorded hand positions and movements. The data captured in Task 1 were analyzed to evaluate HL2’s hand-tracking capabilities against VM. Finger rotation angles from Task 2 and lateral pinch span from Task 3 were then used to assess HL2’s accuracy compared to VM. The results indicate that the HL2 exhibits millimeter-level errors compared to Vicon’s tracking system in Task 1, spanning in a range from 2 mm to 4 mm, suggesting that HL2’s hand-tracking system demonstrates good accuracy. Additionally, the reconstructed grasping positions in Task 2 from both systems show a strong correlation and an average error of 5°, while in Task 3, the accuracy of the HL2 is comparable to that of VM, improving performance as the object thickness increases.