Distributed computing, e.g., cluster and cloud computing, has been applied in almost all areas for data processing, while high resource efficiency and user satisfaction are still the ambition of distributed computing. Task scheduling is indispensable for achieving the goal. As the task scheduling problem is NP-hard, heuristics and meta-heuristics are frequently applied. Every method has its own advantages and limitations. Thus, in this paper, we designed a hybrid heuristic task scheduling problem by exploiting the high global search ability of the Genetic Algorithm (GA) and the fast convergence of Particle Swarm Optimization (PSO). Different from existing hybrid heuristic approaches that simply sequentially perform two or more algorithms, the PGA applies the evolutionary method of a GA and integrates self- and social cognitions into the evolution. We conduct extensive simulated environments for the performance evaluation, where simulation parameters are set referring to some recent related works. Experimental results show that the PGA has 27.9–65.4% and 33.8–69.6% better performance than several recent works, on average, in user satisfaction and resource efficiency, respectively.

In this paper, we focus on the edge server placement problem (ESPP). ESPP is to decide positions (edge stations) where purchased edge servers (ESs) are placed when a service provider builds or upgrades its edge computing solution. The decision of ESPP largely determines the overall service quality and resource efficiency. We aim at providing ESPP solutions for optimizing the service quality in terms of the system response time, given a set of purchased ESs. To achieve this aim, we first deduce the optimal request dispatch for each edge station based on queue theory and Binomial theorem. Then, based on the deduced result, we formulate ESPP into a discrete non-linear constrained optimization model. Because ESPP is NP-hard and generally multi-modality, we designed a hybrid meta-heuristic algorithm to combine advantages of particle swarm optimization (PSO) and genetic algorithm (GA) as well as the niching technology, which is named nPGSAO. The basic idea of nPGSAO is grouping similar individuals into a niche and finding the niche best. Then, nPGSAO evolves each individual by integrating the swarm intelligence of PSO into the evolutionary process of GA based on its personal and niching best. To evaluate the performance of nPGSAO, we conducted extensive simulated experiments. The results show that nPGSAO achieves 5.78%–40.4% better response time with the varied ESPP scale, compared to 11 up-to-date algorithms, and has a linear time complexity with the the ESPP scale. These confirm the superior performance and scalability of nPGSAO.

Local overlapping community detection is a hot problem in the field of studying complex networks. It is the process of finding dense clusters based on local network information. This paper proposes a method called local greedy extended dynamic overlapping community detection (GLOD) to address the challenges of detecting high-quality overlapping communities in complex networks. The goal is to improve the accuracy of community detection by considering the dynamic nature of community boundaries and leveraging local network information. The GLOD method consists of several steps. First, a coupling seed is constructed by selecting nodes from blank communities (i.e., nodes not assigned to any community) and their similar neighboring nodes. This seed serves as the starting point for community detection. Next, the seed boundaries are extended by applying multiple community fitness functions. These fitness functions determine the likelihood of nodes belonging to a specific community based on various local network properties. By iteratively expanding the seed boundaries, communities with higher density and better internal structure are formed. Finally, the overlapping communities are merged using an improved version of the Jaccard coefficient, which is a measure of similarity between sets. This step ensures that overlapping nodes between communities are properly identified and accounted for in the final community structure. The proposed method is evaluated using real networks and three sets of LFR (Lancichinetti–Fortunato–Radicchi) networks, which are synthetic benchmark networks widely used in community detection research. The experimental results demonstrate that GLOD outperforms existing algorithms and achieves a 2.1% improvement in the F-score, a community quality evaluation metric, compared to the LOCD framework. It outperforms the best existing LOCD algorithm on the real provenance network. In summary, the GLOD method aims to overcome the limitations of existing community detection algorithms by incorporating local network information, considering overlapping communities, and dynamically adjusting community boundaries. The experimental results suggest that GLOD is effective in improving the quality of community detection in complex networks.

With the rapid development of IoT technology, sensors are widely used for monitoring environmental parameters. The data collected by sensors needs to be stored, and distributed storage systems provide an excellent platform to handle this vast amount of data. To enhance data reliability and reduce storage costs, erasure coding technology can be employed within distributed storage systems. However, the process of recovering lost or damaged data inevitably generates significant cross-rack traffic. In recent years, various batch recovery methods have been designed to improve data recovery speed and reduce cross-rack recovery traffic, but they have limitations in different aspects, such as less consideration for heterogeneous cross-rack bandwidth and insufficient handling of recovery link scheduling. This paper proposes the HBRepair recovery framework, aimed at improving data recovery speed by balancing the recovery load and reducing cross-rack recovery traffic. Firstly, HBRepair strategically selects helper blocks and nodes for storing recovery blocks to find the optimal batch recovery plan. Then, it selectively and rationally schedules recovery links to saturate idle bandwidth resources and avoid network congestion. Experimental results show that by optimizing the use of cross-rack bandwidth, HBRepair can reduce cross-rack recovery time by up to 26.74%, effectively addressing the shortcomings of existing methods.

With the widespread use of distributed systems and the development of big data technology, the storage redundancy, data availability and persistence of distributed systems have become a concern. Traditional multi-copy storage may cause large storage redundancy, thus wasting storage space. The erasure code is considered as the best alternative to the replica strategy. However, the existing methods do not fully consider the access characteristics of files, so that the data with high access popularity are stored using computationally complex erasure codes, resulting in poor parallel read and write performance. Moreover, due to insufficient consideration of file size, storage redundancy cannot be minimized, thus wasting storage space. Therefore, we propose an adaptive hybrid storage strategy based on file access characteristics called FACHS. For cold files, namely the files with low access frequency, we use RS Code (Reed Solomon Code) to store them. RS Code has low computing and storage costs. We use multi-copy and LRC code (local reconstruction code) to store small and large size hot files respectively. Multi-copy ensures the efficiency of file read/write, as well as the ability of parallel read/write. The recovery cost of LRC code in case of node failure is very low. The experimental results show that compared with the existing methods, FACHS can reduce the storage space occupation by 12% for cold files, and improve the read/write speed by 8% and the recovery efficiency by 29% for hot files.

Erasure codes are widely used in large-scale distributed storage systems due to their high efficiency and reliability, but they also face extremely high repair penalties when data corruption occurs. At present, machine learning methods can accurately predict the next failure time and type of machine nodes. Based on this, in order to solve the problem of unnecessary repair traffic caused by temporary failures, as well as the more degraded reads of high-frequency accessed data due to longer failure time of such data in existing repair methods, we propose an Adaptive Classification Predictive Repair method (ACPR) for different fault scenarios. By categorizing the failed blocks into high-risk and low-risk based on the failure type of the soon-to-fail (STF) node and the access heat of STF blocks, ACPR can perform adaptive predictive repair. By quickly repair high-risk blocks to ensure data availability while delaying the repair of low-risk blocks, a large amount of unnecessary repair traffic caused by temporary node failures in the cluster is avoided. Alibaba Cloud Elastic Compute Service (ECS) experiments results show that compared with FastPR and ECPipe, ACPR can shorten the repair time per data block by up to 15.2% and 33.5%, respectively. Moreover, ACPR can reduce repair traffic by up to 74.1% and 84.4%, respectively.