To quantitatively assess pathological gait, we developed a novel smartphone application for full-body human motion tracking in real time from markerless video-based images using a smartphone monocular camera and deep learning. As training data for deep learning, the original three-dimensional (3D) dataset comprising more than 1 million captured images from the 3D motion of 90 humanoid characters and the two-dimensional dataset of COCO 2017 were prepared. The 3D heatmap offset data consisting of 28 × 28 × 28 blocks with three red–green–blue colors at the 24 key points of the entire body motion were learned using the convolutional neural network, modified ResNet34. At each key point, the hottest spot deviating from the center of the cell was learned using the tanh function. Our new iOS application could detect the relative tri-axial coordinates of the 24 whole-body key points centered on the navel in real time without any markers for motion capture. By using the relative coordinates, the 3D angles of the neck, lumbar, bilateral hip, knee, and ankle joints were estimated. Any human motion could be quantitatively and easily assessed using a new smartphone application named Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) without any body markers or multipoint cameras.

Distinguishing pathological gait is challenging in neurology because of the difficulty of capturing total body movement and its analysis. We aimed to obtain a convenient recording with an iPhone and establish an algorithm based on deep learning. From May 2021 to November 2022 at Yamagata University Hospital, Shiga University, and Takahata Town, patients with idiopathic normal pressure hydrocephalus (n = 48), Parkinson’s disease (n = 21), and other neuromuscular diseases (n = 45) comprised the pathological gait group (n = 114), and the control group consisted of 160 healthy volunteers. iPhone application TDPT-GT captured the subjects walking in a circular path of about 1 meter in diameter, a markerless motion capture system, with an iPhone camera, which generated the three-axis 30 frames per second (fps) relative coordinates of 27 body points. A light gradient boosting machine (Light GBM) with stratified k-fold cross-validation (k = 5) was applied for gait collection for about 1 min per person. The median ability model tested 200 frames of each person’s data for its distinction capability, which resulted in the area under a curve of 0.719. The pathological gait captured by the iPhone could be distinguished by artificial intelligence.

To assess pathological gaits quantitatively, three-dimensional coordinates estimated with a deep learning model were converted into body axis plane projections. First, 15 healthy volunteers performed four gait patterns; that is, normal, shuffling, short-stepped, and wide-based gaits, with the Three-Dimensional Pose Tracker for Gait Test (TDPT-GT) application. Second, gaits of 47 patients with idiopathic normal pressure hydrocephalus (iNPH) and 92 healthy elderly individuals in the Takahata cohort were assessed with the TDPT-GT. Two-dimensional relative coordinates were calculated from the three-dimensional coordinates by projecting the sagittal, coronal, and axial planes. Indices of the two-dimensional relative coordinates associated with a pathological gait were comprehensively explored. The candidate indices for the shuffling gait were the angle range of the hip joint < 30° and relative vertical amplitude of the heel < 0.1 on the sagittal projection plane. For the short-stepped gait, the angle range of the knee joint < 45° on the sagittal projection plane was a candidate index. The candidate index for the wide-based gait was the leg outward shift > 0.1 on the axial projection plane. In conclusion, the two-dimensional coordinates on the body axis projection planes calculated from the 3D relative coordinates estimated by the TDPT-GT application enabled the quantification of pathological gait features.

We aimed to capture the fluctuations in the dynamics of body positions and find the characteristics of them in patients with idiopathic normal pressure hydrocephalus (iNPH) and Parkinson’s disease (PD). With the motion-capture application (TDPT-GT) generating 30 Hz coordinates at 27 points on the body, walking in a circle 1 m in diameter was recorded for 23 of iNPH, 23 of PD, and 92 controls. For 128 frames of calculated distances from the navel to the other points, after the Fourier transforms, the slopes (the representatives of fractality) were obtained from the graph plotting the power spectral density against the frequency in log–log coordinates. Differences in the average slopes were tested by one-way ANOVA and multiple comparisons between every two groups. A decrease in the absolute slope value indicates a departure from the 1/f noise characteristic observed in healthy variations. Significant differences in the patient groups and controls were found in all body positions, where patients always showed smaller absolute values. Our system could measure the whole body’s movement and temporal variations during walking. The impaired fluctuations of body movement in the upper and lower body may contribute to gait and balance disorders in patients.

Pathological gait in patients with Hakim’s disease (HD, synonymous with idiopathic normal-pressure hydrocephalus; iNPH), Parkinson’s disease (PD), and cervical myelopathy (CM) has been subjectively evaluated in this study. We quantified the characteristics of upper and lower limb movements in patients with pathological gait. We analyzed 1491 measurements of 1 m diameter circular walking from 122, 12, and 93 patients with HD, PD, and CM, respectively, and 200 healthy volunteers using the Three-Dimensional Pose Tracker for Gait Test. Upper and lower limb movements of 2D coordinates projected onto body axis sections were derived from estimated 3D relative coordinates. The hip and knee joint angle ranges on the sagittal plane were significantly smaller in the following order: healthy > CM > PD > HD, whereas the shoulder and elbow joint angle ranges were significantly smaller, as follows: healthy > CM > HD > PD. The outward shift of the leg on the axial plane was significantly greater, as follows: healthy < CM < PD < HD, whereas the outward shift of the upper limb followed the order of healthy > CM > HD > PD. The strongest correlation between the upper and lower limb movements was identified in the angle ranges of the hip and elbow joints on the sagittal plane. The lower and upper limb movements during circular walking were correlated. Patients with HD and PD exhibited reduced back-and-forth swings of the upper and lower limbs.