Changes in gait patterns provide important information about individuals’ health. To perform sensor based gait analysis, it is crucial to develop methodologies to automatically segment single strides from continuous movement sequences. In this study we developed an algorithm based on time-invariant template matching to isolate strides from inertial sensor signals. Shoe-mounted gyroscopes and accelerometers were used to record gait data from 40 elderly controls, 15 patients with Parkinson’s disease and 15 geriatric patients. Each stride was manually labeled from a straight 40 m walk test and from a video monitored free walk sequence. A multi-dimensional subsequence Dynamic Time Warping (msDTW) approach was used to search for patterns matching a pre-defined stride template constructed from 25 elderly controls. F-measure of 98% (recall 98%, precision 98%) for 40 m walk tests and of 97% (recall 97%, precision 97%) for free walk tests were obtained for the three groups. Compared to conventional peak detection methods up to 15% F-measure improvement was shown. The msDTW proved to be robust for segmenting strides from both standardized gait tests and free walks. This approach may serve as a platform for individualized stride segmentation during activities of daily living.

Parkinson's disease (PD) is the most frequent neurodegenerative movement disorder. Early diagnosis and effective therapy monitoring is an important prerequisite to treat patients and reduce health care costs. Objective and non-invasive assessment strategies are an urgent need in order to achieve this goal. In this study we apply a mobile, lightweight and easy applicable sensor based gait analysis system to measure gait patterns in PD and to distinguish mild and severe impairment of gait. Examinations of 16 healthy controls, 14 PD patients in an early stage, and 13 PD patients in an intermediate stage were included. Subjects performed standardized gait tests while wearing sport shoes equipped with inertial sensors (gyroscopes and accelerometers). Signals were recorded wirelessly, features were extracted, and distinct subpopulations classified using different classification algorithms. The presented system is able to classify patients and controls (for early diagnosis) with a sensitivity of 88% and a specificity of 86%. In addition it is possible to distinguish mild from severe gait impairment (for therapy monitoring) with 100% sensitivity and 100% specificity. This system may be able to objectively classify PD gait patterns providing important and complementary information for patients, caregivers and therapists.

Handwriting is one of the most frequently occurring patterns in everyday life and with it comes challenging applications such as handwriting recognition, writer identification and signature verification. In contrast to offline HWR that only uses spatial information (i.e., images), online HWR uses richer spatio-temporal information (i.e., trajectory data or inertial data). While there exist many offline HWR datasets, there are only little data available for the development of OnHWR methods on paper as it requires hardware-integrated pens. This paper presents data and benchmark models for real-time sequence-to-sequence learning and single character-based recognition. Our data are recorded by a sensor-enhanced ballpoint pen, yielding sensor data streams from triaxial accelerometers, a gyroscope, a magnetometer and a force sensor at 100 Hz. We propose a variety of datasets including equations and words for both the writer-dependent and writer-independent tasks. Our datasets allow a comparison between classical OnHWR on tablets and on paper with sensor-enhanced pens. We provide an evaluation benchmark for seq2seq and single character-based HWR using recurrent and temporal convolutional networks and transformers combined with a connectionist temporal classification (CTC) loss and cross-entropy (CE) losses. Our convolutional network combined with BiLSTMs outperforms transformer-based architectures, is on par with InceptionTime for sequence-based classification tasks and yields better results compared to 28 state-of-the-art techniques. Time-series augmentation methods improve the sequence-based task, and we show that CE variants can improve the single classification task. Our implementations together with the large benchmark of state-of-the-art techniques of novel OnHWR datasets serve as a baseline for future research in the area of OnHWR on paper.

Objective and rater independent analysis of movement impairment is one of the most challenging tasks in medical engineering. Especially assessment of motor symptoms defines the clinical diagnosis in Parkinson's disease (PD). A sensor-based system to measure the movement of the upper and lower extremities would therefore complement the clinical evaluation of PD. In this study two different sensor-based systems were combined to assess movement of 18 PD patients and 17 healthy controls. First, hand motor function was evaluated using a sensor pen with integrated accelerometers and pressure sensors, and second, gait function was assessed using a sports shoe with attached inertial sensors (gyroscopes, accelerometers). Subjects performed standardized tests for both extremities. Features were calculated from sensor signals to differentiate between patients and controls. For the latter, pattern recognition methods were used and the performance of four classifiers was compared. In a first step classification was done for every single system and in a second step for combined features of both systems. Combination of both motor task assessments substantially improved classification rates to 97% using the AdaBoost classifier for the experiment patients vs. controls. The combination of two different analysis systems led to enhanced, more stable, objective, and rater independent recognition of motor impairment. The method can be used as a complementary diagnostic tool for movement disorders.

Efficient handwriting trajectory reconstruction (TR) requires specific writing surfaces for detecting movements of digital pens. Although several motion-based solutions have been developed to remove the necessity of writing surfaces, most of them are based on classical sensor fusion methods limited, by sensor error accumulation over time, to tracing only single strokes. In this work, we present an approach to map the movements of an IMU-enhanced digital pen to relative displacement data. Training data is collected by means of a tablet. We propose several pre-processing and data-preparation methods to synchronize data between the pen and the tablet, which are of different sampling rates, and train a convolutional neural network (CNN) to reconstruct multiple strokes without the need of writing segmentation or post-processing correction of the predicted trajectory. The proposed system learns the relative displacement of the pen tip over time from the recorded raw sensor data, achieving a normalized error rate of 0.176 relative to unit-scaled tablet ground truth (GT) trajectory. To test the effectiveness of the approach, we train a neural network for character recognition from the reconstructed trajectories, which achieved a character error rate of 19.51%. Finally, a joint model is implemented that makes use of both the IMU data and the generated trajectories, which outperforms the sensor-only-based recognition approach by 0.75%.