Time-of-flight cameras are widely adopted in a variety of indoor applications ranging from industrial object measurement to human activity recognition. However, the available products may differ in terms of the quality of the acquired point cloud, and the datasheet provided by the constructors may not be enough to guide researchers in the choice of the perfect device for their application. Hence, this work details the experimental procedure to assess time-of-flight cameras’ error sources that should be considered when designing an application involving time-of-flight technology, such as the bias correction and the temperature influence on the point cloud stability. This is the first step towards a standardization of the metrological characterization procedure that could ensure the robustness and comparability of the results among tests and different devices. The procedure was conducted on Kinect Azure, Basler Blaze 101, and Basler ToF 640 cameras. Moreover, we compared the devices in the task of 3D reconstruction following a procedure involving the measure of both an object and a human upper-body-shaped mannequin. The experiment highlighted that, despite the results of the previously conducted metrological characterization, some devices showed evident difficulties in reconstructing the target objects. Thus, we proved that performing a rigorous evaluation procedure similar to the one proposed in this paper is always necessary when choosing the right device.

This paper presents the validation of a marker-less motion capture system used to evaluate the upper limb stress of subjects using exoskeletons for locomotion. The system fuses the human skeletonization provided by commercial 3D cameras with forces exchanged by the user to the ground through upper limbs utilizing instrumented crutches. The aim is to provide a low cost, accurate, and reliable technology useful to provide the trainer a quantitative evaluation of the impact of assisted gait on the subject without the need to use an instrumented gait lab. The reaction forces at the upper limbs’ joints are measured to provide a validation focused on clinically relevant quantities for this application. The system was used simultaneously with a reference motion capture system inside a clinical gait analysis lab. An expert user performed 20 walking tests using instrumented crutches and force platforms inside the observed volume. The mechanical model was applied to data from the system and the reference motion capture, and numerical simulations were performed to assess the internal joint reaction of the subject’s upper limbs. A comparison between the two results shows a root mean square error of less than 2% of the subject’s body weight.

This article proposes a state-of-the-art algorithm for estimating the 3D volume enclosed in a surface point cloud via a modified extension of the Monte Carlo integration approach. The algorithm consists of a pre-processing of the surface point cloud, a sequential generation of points managed by an affiliation criterion, and the final computation of the volume. The pre-processing phase allows a spatial re-orientation of the original point cloud, the evaluation of the homogeneity of its points distribution, and its enclosure inside a rectangular parallelepiped of known volume. The affiliation criterion using the explosion of cube faces is the core of the algorithm, handles the sequential generation of points, and proposes the effective extension of the traditional Monte Carlo method by introducing its applicability to the discrete domains. Finally, the final computation estimates the volume as a function of the total amount of generated points, the portion enclosed within the surface point cloud, and the parallelepiped volume. The developed method proves to be accurate with surface point clouds of both convex and concave solids reporting an average percentage error of less than 7 %. It also shows considerable versatility in handling clouds with sparse, homogeneous, and sometimes even missing points distributions. A performance analysis is presented by testing the algorithm on both surface point clouds obtained from meshes of virtual objects as well as from real objects reconstructed using reverse engineering techniques.

Human identification and tracking using ultra-wideband-vision data fusion in unstructured environments

Any severe motor disability is a condition that limits the ability to interact with the environment, even the domestic one, caused by the loss of control over one’s mobility. This work presents RoboEYE, a power wheelchair designed to allow users to move easily and autonomously within their homes. To achieve this goal, an innovative, cost-effective and user-friendly control system was designed, in which a non-invasive eye tracker, a monitor, and a 3D camera represent some of the core elements. RoboEYE integrates functionalities from the mobile robotics field into a standard power wheelchair, with the main advantage of providing the user with two driving options and comfortable navigation. The most intuitive and direct modality foresees the continuous control of frontal and angular wheelchair velocities by gazing at different areas of the monitor. The second, semi-autonomous modality allows navigation toward a selected point in the environment by just pointing and activating the wished destination while the system autonomously plans and follows the trajectory that brings the wheelchair to that point. The purpose of this work was to develop the control structure and driving interface designs of the aforementioned driving modalities taking into account also uncertainties in gaze detection and other sources of uncertainty related to the components to ensure user safety. Furthermore, the driving modalities, in particular the semi-autonomous one, were modeled and qualified through numerical simulations and experimental verification by testing volunteers, who are regular users of standard electric wheelchairs, to verify the efficiency, reliability and safety of the proposed system for domestic use. RoboEYE resulted suitable for environments with narrow passages wider than 1 m, which is comparable with a standard domestic door and due to its properties with large commercialization potential.

In rehabilitation settings that exploit Mixed Reality, a clinician risks losing empathy with the patient by being immersed in different worlds, either real and/or virtual. While the patient perceives the rehabilitation stimuli in a mixed real–virtual world, the physician is only immersed in the real part. While in rehabilitation, this may cause the impossibility for the clinician to intervene, in skill assessment, this may cause difficulty in evaluation. To overcome the above limitation, we propose an innovative Augmented Reality (AR) framework for rehabilitation and skill assessment in clinical settings. Data acquired by a distributed sensor network are used to feed a “shared AR” environment so that both therapists and end-users can effectively operate/perceive it, taking into account the specific interface requirements for each user category: (1) for patients, simplicity, immersiveness, engagement and focus on the task; (2) for clinicians/therapists, contextualization and natural interaction with the whole set of data that is linked with the users’ performances in real-time. This framework has a strong potential in Occupational Therapy (OT) but also in physical, psychological, and neurological rehabilitation. Hybrid real and virtual environments may be quickly developed and personalized to match end users’ abilities and emotional and physiological states and evaluate nearly all relevant performances, thus augmenting the clinical eye of the therapist and the clinician-patient empathy. In this paper, we describe a practical exploitation of the proposed framework in OT: setting-up the table for eating. Both a therapist and a user wear Microsoft HoloLens 2. First, the therapist sets up the table with virtual furniture. Next, the user places the corresponding real objects (also in shape) to match them as closely as possible to the corresponding virtual ones. The therapist’s view is augmented during the test with motion, balance, and physiological estimated cues. Once the training is completed, he automatically perceives deviations in the position and attitude of each object and the elapsed time. We used a camera-based localization algorithm achieving a level of accuracy of 5 mm with a confidence level of 95% for position and 1° for rotation. The framework was designed and tested in collaboration with clinical experts of Villa Rosa rehabilitation hospital in Pergine (Italy), involving both a set of patients and healthy users to demonstrate the effectiveness of the designed architecture and the significance of the analyzed parameters between healthy users and patients.