Over the past few years, the use of swarms of Unmanned Aerial Vehicles (UAVs) in monitoring and remote area surveillance applications has become widespread thanks to the price reduction and the increased capabilities of drones. The drones in the swarm need to cooperatively explore an unknown area, in order to identify and monitor interesting targets, while minimizing their movements. In this work, we propose a distributed Reinforcement Learning (RL) approach that scales to larger swarms without modifications. The proposed framework relies on the possibility for the UAVs to exchange some information through a communication channel, in order to achieve context-awareness and implicitly coordinate the swarm’s actions. Our experiments show that the proposed method can yield effective strategies, which are robust to communication channel impairments, and that can easily deal with non-uniform distributions of targets and obstacles. Moreover, when agents are trained in a specific scenario, they can adapt to a new one with minimal additional training. We also show that our approach achieves better performance compared to a computationally intensive look-ahead heuristic.

The Network Slicing (NS) paradigm enables the partition of physical and virtual resources among multiple logical networks, possibly managed by different tenants. In such a scenario, network resources need to be dynamically allocated according to the slice requirements. In this paper, we attack the above problem by exploiting a Deep Reinforcement Learning approach. Our framework is based on a distributed architecture, where multiple agents cooperate towards a common goal. The agent training is carried out following the Advantage Actor Critic algorithm, which permits to handle continuous action spaces. By means of extensive simulations, we show that our approach yields better performance than both a static allocation of system resources and an efficient empirical strategy. At the same time, the proposed system ensures high adaptability to different scenarios without the need for additional training.

The automation of factories and manufacturing processes has been accelerating over the past few years, leading to an ever-increasing number of scenarios with networked agents whose coordination requires reliable wireless communication. In this context, goal-oriented communication adapts transmissions to the control task, prioritizing the more relevant information to decide which action to take. Instead, networked control models follow the opposite pathway, optimizing physical actions to address communication impairments. In this work, we propose a joint design that combines goal-oriented communication and networked control into a single optimization model, an extension of a multi-agent Partially Observable Markov Decision Process (POMDP), which we call Cyber-Physical POMDP. The proposed model is flexible enough to represent a large variety of scenarios and we illustrate its potential in two simple use cases with a single agent and a set of supporting sensors. Our results assess that the joint optimization of communication and control tasks radically improves the performance of networked control systems, particularly in the case of constrained resources, leading to implicit coordination of communication actions.

The 12-lead electrocardiogram (ECG) is an integral component to the diagnosis of a multitude of cardiovascular conditions. It is performed using a complex set of skin surface electrodes, limiting its use outside traditional clinical settings. We developed an artificial intelligence algorithm, trained over 600,000 clinically acquired ECGs, to explore whether fewer leads as input are sufficient to reconstruct a 12-lead ECG. Two limb leads (I and II) and one precordial lead (V3) were required to generate a reconstructed 12-lead ECG highly correlated with the original ECG. An automatic algorithm for detection of ECG features consistent with acute myocardial infarction (MI) performed similarly for original and reconstructed ECGs (AUC = 0.95). When interpreted by cardiologists, reconstructed ECGs achieved an accuracy of 81.4 ± 5.0% in identifying ECG features of ST-segment elevation MI, comparable with the original 12-lead ECGs (accuracy 84.6 ± 4.6%). These results will impact development efforts to innovate ECG acquisition methods with simplified tools in non-specialized settings.

The most critical burden for People with Epilepsy (PwE) is represented by seizures, the unpredictability of which severely impacts quality of life. The design of real-time warning systems that can detect or even predict ictal events would enhance seizure management, leading to high benefits for PwE and their caregivers. In the past, various research works highlighted that seizure onset is anticipated by significant changes in autonomic cardiac control, which can be assessed through heart rate variability (HRV). This manuscript conducted a scoping review of the literature analyzing HRV-based methods for detecting or predicting ictal events. An initial search on the PubMed database returned 402 papers, 72 of which met the inclusion criteria and were included in the review. These results suggest that seizure detection is more accurate in neonatal and pediatric patients due to more significant autonomic modifications during the ictal transitions. In addition, conventional metrics are often incapable of capturing cardiac autonomic variations and should be replaced with more advanced methodologies, considering non-linear HRV features and machine learning tools for processing them. Finally, studies investigating wearable systems for heart monitoring denoted how HRV constitutes an efficient biomarker for seizure detection in patients presenting significant alterations in autonomic cardiac control during ictal events.

Pragmatic or goal-oriented communication can optimize communication decisions beyond the reliable transmission of data, instead aiming at directly affecting application performance with the minimum channel utilization. In this paper, we develop a general theoretical framework for the remote control of finite-state Markov processes, using pragmatic communication over a costly zero-delay communication channel. To that end, we model a cyber-physical system composed of an encoder, which observes and transmits the states of a process in real-time, and a decoder, which receives that information and controls the behavior of the process. The encoder and the decoder should cooperatively optimize the trade-off between the control performance (i.e., reward) and the communication cost (i.e., channel use). This scenario underscores a pragmatic (i.e., goal-oriented) communication problem, where the purpose is to convey only the data that is most valuable for the underlying task, taking into account the state of the decoder (hence, the pragmatic aspect). We investigate two different decision-making architectures: in pull-based remote control, the decoder is the only decision-maker, while in push-based remote control, the encoder and the decoder constitute two independent decision-makers, leading to a multi-agent scenario. We propose three algorithms to optimize our system (i.e., design the encoder and the decoder policies), discuss the optimality guarantees ofs the algorithms, and shed light on their computational complexity and fundamental limits.