This study introduces a multi-agent reinforcement learning approach to address the challenges of real-time scheduling in dynamic environments, with a specific focus on healthcare operations. The proposed system integrates the Human-in-the-Loop (HITL) paradigm, providing continuous feedback from human evaluators, and it employs a sophisticated reward function to attenuate the effects of human-driven events. Novel mapping between reinforcement learning (RL) concepts and the Belief–Desire–Intention (BDI) framework is developed to enhance the explainability of the agent’s decision-making. A system is designed to adapt to changes in patient conditions and preferences while minimizing disruptions to existing schedules. Experimental results show a notable decrease in patient waiting times compared to conventional methods while adhering to operator-induced constraints. This approach offers a robust, explainable, and adaptable solution for the challenging tasks of scheduling in the environments that require human-centered decision-making.
1 June 2023 in Scientific and Technical Journal of Information Technologies, Mechanics and Optics
The ways of substantiating the clinical decision of doctors in the absence of clinical treatment protocols are considered. A comparative evaluation of various statistical methods for ranking clinical symptoms in terms of significance for predicting the outcome of the disease in a small sample of patients with COVID-19 and a history of cardiovascular diseases was performed. The data set (141 patients, 81 factors) was formed based on the materials of electronic medical records of patients of the Federal State Budgetary Institution “National Medical Research Center named after V.A. Almazov”. A subset of controllable risk factors (51 factors) was identified. Descriptive statistics methods (one-way ANOVA, Mann-Whitney and χ² tests) and dimensionality reduction methods (univariate linear regression combined with multiple logistic regression, generalized discriminant analysis, and various decision tree algorithms) were used to rank the factors. To compare the ranking results and evaluate the statistical stability, Kendall’s correlation was used, visualized as a heat map and a positional graph. It has been established that the use of descriptive statistics methods is justified when ranking on a small sample size of patients. It is shown that the ensemble of ranking results may be statistically inconsistent. It is concluded that the positions of the same features obtained by ranking them as part of a complete set and a subset of features do not match; therefore, when choosing a statistical processing method for expert evaluation, one should take into account the meaningful formulation of the problem. It is shown that the statistical stability of ranking under conditions of small samples depends on the number of features taken into account, and this dependence is significantly different for different ranking methods. The proposed method of intellectual support and verification of clinical decisions in terms of choosing the most significant clinical signs can be used to select and justify the tactics of managing patients in the absence of clinical protocols.
This paper presents a neuro-symbolic approach for constructing distributed knowledge graphs to facilitate cooperation through communication among spatially proximate agents. We develop a graph autoencoder (GAE) that learns rich representations from heterogeneous modalities. The method employs density-adaptive k-nearest neighbor (k-NN) construction with Gabriel pruning to build the proximity graphs that balance local density awareness with geometric consistency. When the agents enter the bridging zone, their individual knowledge graphs are aggregated into hypergraphs using a construction algorithm, for which we derive the theoretical bounds on the minimum number of hyperedges required for connectivity under arity and locality constraints. We evaluate the approach in PettingZoo’s communication-oriented environment, observing improvements of approximately 10% in episode rewards and up to 40% in individual agent rewards compared to Deep Q-Network (DQN) baselines, while maintaining comparable policy loss values. The explicit graph structures may offer interpretability benefits for applications requiring auditability. This work explores how structured knowledge representations can support cooperation in distributed multi-agent systems with heterogeneous observations.
1 December 2023 in Scientific and Technical Journal of Information Technologies, Mechanics and Optics
For the first time, a simplified approach to constructing Bayesian neural networks is proposed, combining computational efficiency with the ability to analyze the learning process. The proposed approach is based on Bayesianization of a deterministic neural network by randomizing parameters only at the interface level, i.e., the formation of a Bayesian neural network based on a given network by replacing its parameters with probability distributions that have the parameters of the original model as the average value. Evaluations of the efficiency metrics of the neural network were obtained within the framework of the approach under consideration, and the Bayesian neural network constructed through variation inference were performed using topological data analysis methods. The Bayesianization procedure is implemented through graded variation of the randomization intensity. As an alternative, two neural networks with identical structure were used — deterministic and classical Bayesian networks. The input of the neural network was supplied with the original data of two datasets in versions without noise and with added Gaussian noise. The zero and first persistent homologies for the embeddings of the formed neural networks on each layer were calculated. To assess the quality of classification, the accuracy metric was used. It is shown that the barcodes for embeddings on each layer of the Bayesianized neural network in all four scenarios are between the corresponding barcodes of the deterministic and Bayesian neural networks for both zero and first persistent homologies. In this case, the deterministic neural network is the lower bound, and the Bayesian neural network is the upper bound. It is shown that the structure of data associations within a Bayesianized neural network is inherited from a deterministic model, but acquires the properties of a Bayesian one. It has been experimentally established that there is a relationship between the normalized persistent entropy calculated on neural network embeddings and the accuracy of the neural network. For predicting accuracy, the topology of embeddings on the middle layer of the neural network model turned out to be the most revealing. The proposed approach can be used to simplify the construction of a Bayesian neural network from an already trained deterministic neural network, which opens up the possibility of increasing the accuracy of an existing neural network without ensemble with additional classifiers. It becomes possible to proactively evaluate the effectiveness of the generated neural network on simplified data without running it on a real dataset, which reduces the resource intensity of its development.