When patients perform musculoskeletal rehabilitation exercises, it is of great importance to observe the correctness of their performance. The aim of this study is to increase the accuracy of recognizing human movements during exercise. The process of monitoring and evaluating musculoskeletal rehabilitation exercises was modeled using various tracking systems, and the necessary algorithms for processing information for each of the tracking systems were formalized. An approach to classifying exercises using machine learning methods is presented. Experimental studies were conducted to identify the most accurate tracking systems (virtual reality trackers, motion capture, and computer vision). A comparison of machine learning models is carried out to solve the problem of classifying musculoskeletal rehabilitation exercises, and 96% accuracy is obtained when using multilayer dense neural networks. With the use of computer vision technologies and the processing of a full set of body points, the accuracy of classification achieved is 100%. The hypotheses on the ranking of tracking systems based on the accuracy of positioning of human target points, the presence of restrictions on application in the field of musculoskeletal rehabilitation, and the potential to classify exercises are fully confirmed.

In virtual reality (VR) systems, a problem is the accurate reproduction of the user’s body in a virtual environment using inverse kinematics because existing motion capture systems have a number of drawbacks, and minimizing the number of key tracking points (KTPs) leads to a large error. To solve this problem, it is proposed to use the concept of a digital shadow and machine learning technologies to optimize the number of KTPs. A technique for movement process data collecting from a virtual avatar is implemented, modeling of nonlinear dynamic processes of human movement based on a digital shadow is carried out, the problem of optimizing the number of KTP is formulated, and an overview of the applied machine learning algorithms and metrics for their evaluation is given. An experiment on a dataset formed from virtual avatar movements shows the following results: three KTPs do not provide sufficient reconstruction accuracy, the choice of five or seven KTPs is optimal; among the algorithms, the most efficient in descending order are AdaBoostRegressor, LinearRegression, and SGDRegressor. During the reconstruction using AdaBoostRegressor, the maximum deviation is not more than 0.25 m, and the average is not more than 0.10 m.

The solution to the problem of insufficient accuracy in determining the position and speed of human movement during interaction with a treadmill-based training complex is considered. Control command generation based on the training complex user’s actions may be performed with a delay, may not take into account the specificity of movements, or be inaccurate due to the error of the initial data. The article introduces a technology for improving the accuracy of predicting a person’s position and speed on a running platform using machine learning and computer vision methods. The proposed technology includes analysing and processing data from the tracking system, developing machine learning models to improve the quality of the raw data, predicting the position and speed of human movement, and implementing and integrating neural network methods into the running platform control system. Experimental results demonstrate that the decision tree (DT) model provides better accuracy and performance in solving the problem of positioning key points of a human model in complex conditions with overlapping limbs. For speed prediction, the linear regression (LR) model showed the best results when the analysed window length was 10 frames. Prediction of the person’s position (based on 10 previous frames) is performed using the DT model, which is optimal in terms of accuracy and computation time relative to other options. The comparison of the control methods of the running platform based on machine learning models showed the advantage of the combined method (linear control function combined with the speed prediction model), which provides an average absolute error value of 0.116 m/s. The results of the research confirmed the achievement of the primary objective (increasing the accuracy of human position and speed prediction), making the proposed technology promising for application in human-machine systems.

The article deals with the problem of forming a digital shadow of the process of moving a person. An analysis of the subject area was carried out, which showed the need to formalize the process of creating digital shadows to simulate human movements in virtual space, testing software and hardware systems that operate on the basis of human actions, as well as in various systems of musculoskeletal rehabilitation. It was revealed that among the existing approaches to the capture of human movements, it is impossible to single out a universal and stable method under various environmental conditions. A method for forming a digital shadow has been developed based on combining and synchronizing data from three motion capture systems (virtual reality trackers, a motion capture suit, and cameras using computer vision technologies). Combining the above systems makes it possible to obtain a comprehensive assessment of the position and condition of a person regardless of environmental conditions (electromagnetic interference, illumination). To implement the proposed method, a formalization of the digital shadow of the human movement process was carried out, including a description of the mechanisms for collecting and processing data from various motion capture systems, as well as the stages of combining, filtering, and synchronizing data. The scientific novelty of the method lies in the formalization of the process of collecting data on the movement of a person, combining and synchronizing the hardware of the motion capture systems to create digital shadows of the process of moving a person. The obtained theoretical results will be used as a basis for software abstraction of a digital shadow in information systems to solve the problems of testing, simulating a person, and modeling his reaction to external stimuli by generalizing the collected data arrays about his movement. В статье рассматривается задача формирования цифровой тени процесса перемещения человека. Проведен анализ предметной области, который показал необходимость формализации процесса создания цифровых теней для имитации движений человека в виртуальном пространстве, тестировании программно-аппаратных комплексов, функционирующих на основе действий человека, а также в различных системах опорно-двигательной реабилитации. Выявлено, что среди существующих подходов к захвату движений человека нельзя выделить универсальный и стабильно работающий при различных условиях внешней среды. Разработан метод формирования цифровой тени на основе комбинирования и синхронизации данных из трех систем захвата движений (трекеры виртуальной реальности, костюм motion capture и камеры с использованием технологий компьютерного зрения). Объединение перечисленных систем позволяет получить комплексную оценку положения и состояния человека независимо от условий внешней среды (электромагнитные помехи, освещенность). Для реализации предложенного метода проведена формализация цифровой тени процесса перемещения человека, включающая описание механизмов сбора и обработки данных от различных систем захвата движений, а также этапы объединения, фильтрации и синхронизации данных. Научная новизна метода заключается в формализации процесса сбора данных о перемещении человека, объединении и синхронизации аппаратного обеспечения используемых систем захвата движений для создания цифровых теней процесса перемещения человека. Полученные теоретические результаты будут использоваться в качестве основы для программной абстракции цифровой тени в информационных системах для решения задач тестирования, имитации человека и моделирования его реакции на внешние раздражители за счет обобщения собранных массивов данных о его перемещении.

In the organizing of professional training, the assessment of the trainee’s reaction and state in stressful situations is of great importance. Phobic reactions are a specific type of stress reaction that, however, is rarely taken into account when developing virtual simulators, and are a risk factor in the workplace. A method for evaluating the impact of various phobic stimuli on the quality of training is considered, which takes into account the time, accuracy, and speed of performing professional tasks, as well as the characteristics of electroencephalograms (the amplitude, power, coherence, Hurst exponent, and degree of interhemispheric asymmetry). To evaluate the impact of phobias during experimental research, participants in the experimental group performed exercises in different environments: under normal conditions and under the influence of acrophobic and arachnophobic stimuli. The participants were divided into subgroups using clustering algorithms and an expert neurologist. After that, a comparison of the subgroup metrics was carried out. The research conducted makes it possible to partially confirm our hypotheses about the negative impact of phobic effects on some participants in the experimental group. The relationship between the reaction to a phobia and the characteristics of brain activity was revealed, and the characteristics of the electroencephalogram signal were considered as the metrics for detecting a phobic reaction.

The task of automating and reducing the complexity of the process of developing virtual training complexes is considered. The analysis of the subject area showed the need to move from a monolithic to a service-oriented version of the architecture. It is found that the use of a monolithic architecture in the implementation of virtual training complexes limits the possibility of modernizing the system, increases its software complexity, and makes it difficult to implement an interface for managing and monitoring the training process. The general concept of the microservice architecture of virtual training complexes is presented, and definitions of the main and secondary components are given. The scientific novelty of the research lies in the transition from the classical monolithic architecture in the subject area of the HTC to the microservice architecture; eliminating the shortcomings of this approach by implementing a single protocol for the exchange of information between modules; separation of network interaction procedures into software libraries to unify and improve the reliability of the system. The use of isolated, loosely coupled microservices allows developers to use the best technologies, platforms and frameworks for their implementation; separate the graphical interface of the simulator instructor from the visualization and virtual reality system; provide the ability to flexibly replace the main components (visualization, interface, interaction with virtual reality) without changing the architecture and affecting other modules. The decomposition of the structural model of the microservice architecture is carried out, and the specifics of the functioning of the main components are presented. The implementation of microservices networking libraries and a JSON-based data exchange protocol is considered. The practical significance of the proposed architecture lies in the possibility of parallelization and reducing the complexity of the development and modernization of training complexes. The features of the functioning of the systems implemented in the proposed microservice architecture are analyzed.