Big Data analytics for storing, processing, and analyzing large-scale datasets has become an essential tool for the industry. The advent of distributed computing frameworks such as Hadoop and Spark offers efficient solutions to analyze vast amounts of data. Due to the application programming interface (API) availability and its performance, Spark becomes very popular, even more popular than the MapReduce framework. Both these frameworks have more than 150 parameters, and the combination of these parameters has a massive impact on cluster performance. The default system parameters help the system administrator deploy their system applications without much effort, and they can measure their specific cluster performance with factory-set parameters. However, an open question remains: can new parameter selection improve cluster performance for large datasets? In this regard, this study investigates the most impacting parameters, under resource utilization, input splits, and shuffle, to compare the performance between Hadoop and Spark, using an implemented cluster in our laboratory. We used a trial-and-error approach for tuning these parameters based on a large number of experiments. In order to evaluate the frameworks of comparative analysis, we select two workloads: WordCount and TeraSort. The performance metrics are carried out based on three criteria: execution time, throughput, and speedup. Our experimental results revealed that both system performances heavily depends on input data size and correct parameter selection. The analysis of the results shows that Spark has better performance as compared to Hadoop when data sets are small, achieving up to two times speedup in WordCount workloads and up to 14 times in TeraSort workloads when default parameter values are reconfigured.

Lung cancer is one of the fatal diseases whose early diagnosis is essential to mitigate the death rate. Computed Tomography (CT) scans are widely used for lung cancer diagnosis, but manual interpretation by health professionals can lead to inconsistent results. To address this, we propose Lung-AttNet, a novel lightweight convolutional neural network (CNN) model enhanced with an attention mechanism. Lung-AttNet incorporates a convolutional block with a Lightweight Global Attention Module (LGAM) to effectively distinguish between lung cancer types. The convolutional block extracts both low- and high-dimensional features, while LGAM captures feature dependencies across channel and spatial dimensions. The model is evaluated using the Kaggle CT scan dataset, which includes four classes: adenocarcinoma, large cell carcinoma, squamous cell carcinoma, and normal. Extensive experiments, including ablation studies, 5-fold cross-validation, and explainable AI (XAI) techniques such as Grad-CAM and LIME, demonstrate that Lung-AttNet achieves an average accuracy of 91.5%. Furthermore, to address medical data sensitivity and privacy concerns, the model is deployed in a Federated Learning (FL) framework, where the global model is trained using weights from local models rather than sharing raw data. In the FL environment, Lung-AttNet achieves an accuracy of 92% with 2 and 3 clients, underscoring its robustness and adaptability for real-world applications.

Due to the rapid growth of available data, various platforms offer parallel infrastructure that efficiently processes big data. One of the critical issues is how to use these platforms to optimise resources, and for this reason, performance prediction has been an important topic in the last few years. There are two main approaches to the problem of predicting performance. One is to fit data into an equation based on a analytical models. The other is to use machine learning (ML) in the form of regression algorithms. In this paper, we have investigated the difference in accuracy for these two approaches. While our experiments used an open-source platform called Apache Spark, the results obtained by this research are applicable to any parallel platform and are not constrained to this technology. We found that gradient boost, an ML regressor, is more accurate than any of the existing analytical models as long as the range of the prediction follows that of the training. We have investigated analytical and ML models based on interpolation and extrapolation methods with k-fold cross-validation techniques. Using the interpolation method, two analytical models, namely 2D-plate and fully-connected models, outperform older analytical models and kernel ridge regression algorithm but not the gradient boost regression algorithm. We found the average accuracy of 2D-plate and fully-connected models using interpolation are 0.962 and 0.961. However, when using the extrapolation method, the analytical models are much more accurate than the ML regressors, particularly two of the most recently proposed models (2D-plate and fully-connected). Both models are based on the communication patterns between the nodes. We found that using extrapolation, kernel ridge, gradient boost and two proposed analytical models average accuracy is 0.466, 0.677, 0.975, and 0.981, respectively. This study shows that practitioners can benefit from analytical models by being able to accurately predict the runtime outside of the range of the training data using only a few experimental operations.

This study proposes a dielectric terahertz (THz) porous core fibre with ultra‐high birefringence and near zero dispersion‐flattened properties. The finite element method is used to design and analyse properties of the proposed THz fibre. The proposed porous core crystal fibre has a triangular lattice with microstructured circular air holes in the outer cladding and elliptical air holes in the core. The design exhibits a high birefringence of 7 × 10 ⁻² and a low effective material loss of 0.1 cm ⁻¹ at the operating frequency of f = 1 THz. It also shows nearly zero flattened dispersion with an absolute dispersion variation of ±0.05 ps/THz/cm in the frequency range of 1.3 to 2.2 THz. Power fraction and confinement loss are also reported in this study. The proposed THz porous fibre is deemed suitable for polarisation‐maintaining THz wave guidance.