This study focuses on Digital Twin-integrated smart energy systems, which serve as an example of Next-Generation Critical Infrastructures (CI). The resilience of these systems is influenced by a variety of internal features and external interactions, all of which are subject to change following cyber-physical disturbances. This necessitates real-time resilience monitoring for CI during crises; however, a significant gap remains in resilience monitoring. To address this gap, this study leverages the role of Internet of Things (IoT) in monitoring complex systems to enhance resilience through critical indicators relevant to cyber-physical safety and security. The study empirically implements Resilience-Key Performance Indicators (R-KPIs) from the domain, including Functionality Loss, Minimum Performance, and Recovery Time Duration. The main goal is to examine real-time IoT-based resilience monitoring in a real-life context. A cyber-physical system equipped with IoT-driven Digital Twins, data-driven microservices, and a False Data Injection Attack (FDIA) scenario is simulated to assess the real-time resilience of this smart system. The results demonstrate that real-time resilience monitoring provides actionable insights into resilience performance based on the selected R-KPIs. These findings contribute to a systematic and reusable model for enhancing the resilience of IoT-enabled CI, advancing efforts to ensure service continuity and secure essential services for society.
This paper aims to define design patterns specifically for data ingestion techniques within cloud-based architectures, addressing the challenges associated with high-volume data processing. The approach utilizes a flexible, metadata-driven framework that enhances adaptability and ease of use. This framework supports both incremental and full refresh methods, allowing for seamless changes to ingestion types, schema updates, table additions, and the incorporation of new data sources with minimal intervention from data engineers. The proposed design patterns were validated through experiments conducted on the Azure and Google Cloud platforms. The experiments demonstrate that the proposed design patterns significantly reduce data ingestion time, showcasing their effectiveness in managing high-volume data ingestion. This paper contributes to the field of data management by presenting a comprehensive definition of design patterns tailored for data ingestion in cloud-based architectures, effectively addressing key challenges in high-volume data processing.
Data literacy and the fundamentals of big data management are becoming interdisciplinary in Higher Education curricula, also due to the widespread need of data science skills. This casts the need for presenting novel (and more engaging) learning activities to students. Data hackathons (also known as datathons) represent a viable option to allow students practicing with real use cases and datasets, as well as addressing their learning experiences collaboratively. Moreover, datathons promise to improve soft skills and offer hands-on learning opportunities. Therefore, we present in this paper a datathon on a publicly available dataset about renewable energy systems. The datathon involved students from three data-focused courses of three different M.S. degrees at the University of Salento (Italy). A detailed analysis of the design, implementation and evaluation choices is proposed, along with a series of gathered insights and lessons learned that might help systematizing the introduction and use of datathons in data education.
The rapid growth of data has created significant challenges in managing and leveraging data effectively. Data engineering has emerged as a crucial discipline to address these challenges, providing frameworks for efficient data management. Data Engineering Patterns (DEP) and Data Engineering Design Patterns (DEDP) offer standardized practices and best practice solutions for data engineering tasks like ETL. While various DEPs and DEDPs exist, the issue of high-volume data ingestion remains insufficiently addressed. This paper focuses on defining design patterns specifically for data ingestion techniques within cloud-based architectures, covering both incremental and full refresh methods. The proposed approach utilizes a flexible, metadata-driven framework to enhance adaptability and ease of use, allowing for seamless changes to the ingestion type, schema updates, table additions, and incorporation of new data sources. Validated on the Azure cloud platform, the experiments demonstrate that the proposed design patterns significantly reduce data ingestion time, contributing to the field of data management by addressing key challenges in high-volume data processing.