The exponential growth of data coupled with the widespread application of artificial intelligence(AI) presents organizations with challenges in upholding data accuracy, especially within data engineering functions. While the Extraction, Transformation, and Loading process addresses error-free data ingestion, validating the content within data streams remains a challenge. Prompt detection and remediation of data issues are crucial, especially in automated analytical environments driven by AI. To address these issues, this study focuses on detecting drifts in data distributions and divergence within data fields processed from different sample populations. Using a hypothetical banking scenario, we illustrate the impact of data drift on automated decision-making processes. We propose a scalable method leveraging the Kullback-Leibler (KL) divergence measure, specifically the Population Stability Index (PSI), to detect and quantify data drift. Through comprehensive simulations, we demonstrate the effectiveness of PSI in identifying and mitigating data drift issues. This study contributes to enhancing data engineering functions in organizations by offering a scalable solution for early drift detection in data ingestion pipelines. We discuss related research works, identify gaps, and present the methodology and experiment results, underscoring the importance of robust data governance practices in mitigating risks associated with data drift and improving data observability.

The Nile Delta major cities, particularly Cairo, experienced stagnant air pollution episodes, known as Black Cloud, every year over the past decade during autumn. Low-elevated thermal inversion layers play a crucial role in intensifying pollution impacts. Carbon monoxide, ozone, atmospheric temperature, water vapor, and methane measurements from the tropospheric emission spectrometer (TES) on board the Aura have been used to assess the dominant component below the inversion layer. In this study, time series analysis, autocorrelations, and cross correlations are performed to gain a better understanding of the connections between those parameters and their local effect. Satellite-based data were obtained for the years 2005–2010. The parameters mentioned were investigated throughout the whole year in order to study the possible episodes that take place in addition to their change from year to year. Ozone and carbon monoxide were the two major indicators to the most basic episodes that occur over Cairo and the Delta region.

Using new mathematical and data-driven techniques, we propose new indices to measure and predict the strength of different El Niño events and how they affect regions like the Nile River Basin (NRB). Empirical Mode Decomposition (EMD), when applied to Southern Oscillation Index (SOI), yields three Intrinsic Mode Functions (IMF) tracking recognizable and physically significant non-stationary processes. The aim is to characterize underlying signals driving ENSO as reflected in SOI, and show that those signals also meaningfully affect other physical processes with scientific and predictive utility. In the end, signals are identified which have a strong statistical relationship with various physical factors driving ENSO variation. IMF 6 is argued to track El Niño and La Niña events occurrence, while IMFs 7 and 8 represent another signal, which reflects on variations in El Niño strength and variability between events. These we represent an underlying inter-annual variation between different El Niño events. Due to the importance of the latter, IMFs 7 and 8, are defined as Interannual ENSO Variability Indices (IEVI) and referred to as IEVI α and IEVI β. EMD when applied to the NRB precipitation, affecting the Blue Nile yield, identifying the IEVI-driven IMFs, with high correlations of up to ρ = 0.864, suggesting a decadal variability within NRB that is principally driven by interannual decadal-scale variability highlighting known geographical relationships. Significant hydrological processes, driving the Blue Nile yield, are accurately identified using the IEVI as a predictor. The IEVI-based model performed significantly atp = 0.038 with Blue Nile yield observations.

In the field of brain tumor segmentation, accurately capturing the complexities of tumor sub-regions poses significant challenges. Traditional segmentation methods usually fail to accurately segment tumor subregions. This research introduces a novel solution employing Graph Neural Networks (GNNs), enriched with spectral and spatial insight. In the supervoxel creation phase, we explored methods like VCCS, SLIC, Watershed, Meanshift, and Felzenszwalb–Huttenlocher, evaluating their performance based on homogeneity, moment of inertia, and uniformity in shape and size. After creating supervoxels, we represented 3D MRI images as a graph structure. In this study, we combined Spatial and Spectral GNNs to capture both local and global information. Our Spectral GNN implementation employs the Laplacian matrix to efficiently map tumor tissue connectivity by capturing the graph’s global structure. Consequently, this enhances the model’s precision in classifying brain tumors into distinct types: necrosis, edema, and enhancing tumor. This model underwent extensive hyper-parameter tuning to ascertain the most effective configuration for optimal segmentation performance. Our Spectral–Spatial GNN model surpasses traditional segmentation methods in accuracy for both whole tumor and sub-regions, validated by metrics such as the dice coefficient and accuracy. For the necrotic core, the Spectral–Spatial GNN model showed a 10.6% improvement over the Spatial GNN and 8% over the Spectral GNN. Enhancing tumor gains were 9.5% and 6.4%, respectively. For edema, improvements were 12.8% over the Spatial GNN and 7.3% over the Spectral GNN, highlighting its segmentation accuracy for each tumor sub-region. This superiority underscores the model’s potential in improving brain tumor segmentation accuracy, precision, and computational efficiency.

We use the computing technology digital image processing (DIP) to enhance the teaching of linear algebra so as to make the course more visual and interesting. Certainly, this visual approach by using technology to link linear algebra to DIP is interesting and unexpected to both students as well as many faculty.