With the increase of complex scientific simulations driven by workflows and heterogeneous workload profiles, managing system resources effectively is essential for improving performance and system throughput, especially due to trends like heterogeneous HPC and deeply integrated systems with on-chip accelerators. For optimal resource utilization, dynamic resource allocation can improve productivity across all system and application levels, by adapting the applications' configurations to the system's resources. In this context, malleable jobs, which can change resources at runtime, can increase the system throughput and resource utilization while bringing various advantages for HPC users (e.g., shorter waiting time). Malleability has received much attention recently, even though it has been an active research area for almost two decades [1]. This paper presents the state-of-the-art of malleable implementations in HPC systems, targeting mainly malleability in compute and I/O resources. Based on our experiences, we state our current concerns and list future opportunities for research.
In HPC platforms, concurrent applications are sharing the same file system. This can lead to conflicts, especially as applications are more and more data intensive. I/O contention can represent a performance bottleneck. The access to bandwidth can be split in two complementary yet distinct problems. The mapping problem and the scheduling problem. The mapping problem consists in selecting the set of applications that are in competition for the I/O resource. The scheduling problem consists then, given I/O requests on the same resource, in determining the order to these accesses to minimize the I/O time. In this work we propose to couple a novel bandwidth-aware mapping algorithm to I/O list-scheduling policies to develop a cross-layer optimization solution. We study this solution experimentally using an I/O middleware: CLARISSE. We show that naive policies such as FIFO perform relatively well in order to schedule I/O movements, and that the important part to reduce congestion lies mostly on the mapping part. We evaluate the algorithm that we propose using a simulator that we validated experimentally. This evaluation shows important gains for the simple, bandwidth-aware mapping solution that we provide compared to its non bandwidth-aware counterpart. The gains are both in terms of machine efficiency (makespan) and application efficiency (stretch). This stresses even more the importance of designing efficient, bandwidth-aware mapping strategies to alleviate the cost of I/O congestion.
Background: To understand how infectious agents disseminate throughout a population it is essential to capture the social model in a realistic manner. This paper presents a novel approach to modeling the propagation of the influenza virus throughout a realistic interconnection network based on actual individual interactions which we extract from online social networks. The advantage is that these networks can be extracted from existing sources which faithfully record interactions between people in their natural environment. We additionally allow modeling the characteristics of each individual as well as customizing his daily interaction patterns by making them time-dependent. Our purpose is to understand how the infection spreads depending on the structure of the contact network and the individuals who introduce the infection in the population. This would help public health authorities to respond more efficiently to epidemics. Results: We implement a scalable, fully distributed simulator and validate the epidemic model by comparing the simulation results against the data in the 2004-2005 New York State Department of Health Report (NYSDOH), with similar temporal distribution results for the number of infected individuals. We analyze the impact of different types of connection models on the virus propagation. Lastly, we analyze and compare the effects of adopting several different vaccination policies, some of them based on individual characteristics -such as age- while others targeting the super-connectors in the social model. Conclusions: This paper presents an approach to modeling the propagation of the influenza virus via a realistic social model based on actual individual interactions extracted from online social networks. We implemented a scalable, fully distributed simulator and we analyzed both the dissemination of the infection and the effect of different vaccination policies on the progress of the epidemics. The epidemic values predicted by our simulator match real data from NYSDOH. Our results show that our simulator can be a useful tool in understanding the differences in the evolution of an epidemic within populations with different characteristics and can provide guidance with regard to which, and how many, individuals should be vaccinated to slow down the virus propagation and reduce the number of infections.
This work presents simulation results for different mitigation and confinement scenarios for the propagation of COVID-19 in the metropolitan area of Madrid. These scenarios were implemented and tested using EpiGraph, an epidemic simulator which has been extended to simulate COVID-19 propagation. EpiGraph implements a social interaction model, which realistically captures a large number of characteristics of individuals and groups, as well as their individual interconnections, which are extracted from connection patterns in social networks. Besides the epidemiological and social interaction components, it also models people's short and long-distance movements as part of a transportation model. These features, together with the capacity to simulate scenarios with millions of individuals and apply different contention and mitigation measures, gives EpiGraph the potential to reproduce the COVID-19 evolution and study medium-term effects of the virus when applying mitigation methods. EpiGraph, obtains closely aligned infected and death curves related to the first wave in the Madrid metropolitan area, achieving similar seroprevalence values. We also show that selective lockdown for people over 60 would reduce the number of deaths. In addition, evaluate the effect of the use of face masks after the first wave, which shows that the percentage of people that comply with mask use is a crucial factor for mitigating the infection's spread.
Background: Predicting the details of how an epidemic evolves is highly valuable as health institutions need to better plan towards limiting the infection propagation effects and optimizing their prediction and response capabilities. Simulation is a cost- and time-effective way of predicting the evolution of the infection as the joint influence of many different factors: interaction patterns, personal characteristics, travel patterns, meteorological conditions, previous vaccination, etc. The work presented in this paper extends EpiGraph, our influenza epidemic simulator, by introducing a meteorological model as a modular component that interacts with the rest of EpiGraph’s modules to refine our previous simulation results. Our goal is to estimate the effects of changes in temperature and relative humidity on the patterns of epidemic influenza based on data provided by the Spanish Influenza Sentinel Surveillance System (SISSS) and the Spanish Meteorological Agency (AEMET). Methods: Our meteorological model is based on the regression model developed by AB and JS, and it is tuned with influenza surveillance data obtained from SISSS. After pre-processing this data to clean it and reconstruct missing samples, we obtain new values for the reproduction number of each urban region in Spain, every 10 minutes during 2011. We simulate the propagation of the influenza by setting the date of the epidemic onset and the initial influenza-illness rates for each urban region. Results: We show that the simulation results have the same propagation shape as the weekly influenza rates as recorded by SISSS. We perform experiments for a realistic scenario based on actual meteorological data from 2010-2011, and for synthetic values assumed under simplified predicted climate change conditions. Results show that a diminishing relative humidity of 10% produces an increment of about 1.6% in the final infection rate. The effect of temperature changes on the infection spread is also noticeable, with a decrease of 1.1% per extra degree.Conclusions: Using a tool like ours could help predict the shape of developing epidemics and its peaks, and would permit to quickly run scenarios to determine the evolution of the epidemic under different conditions. We make EpiGraph source code and epidemic data publicly available.