Cloud computing is an emerging paradigm which provides a flexible and diversified trading market for Infrastructure‐as‐a‐Service (IaaS) providers, Software‐as‐a‐Service (SaaS) providers, and cloud‐based application customers. Taking the perspective of SaaS providers, they offer various SaaS services using rental cloud resources supplied by IaaS providers to their end users. In order to maximize their utility, the best behavioural strategy is to reduce renting expenses as much as possible while providing sufficient processing capacity to meet customer demands. In reality, public IaaS providers such as Amazon offer different types of virtual machine (VM) instances with different pricing models. Moreover, service requests from customers always change as time goes by. In such heterogeneous and changing environments, how to realize application auto‐scaling becomes increasingly significant for SaaS providers. In this paper, we first formulate this problem and then propose a Q‐learning based self‐adaptive renting plan generation approach to help SaaS providers make efficient IaaS facilities adjustment decisions dynamically. Through a series of experiments and simulation, we evaluate the auto‐scaling approach under different market conditions and compare it with two other resource allocation strategies. Experimental results show that our approach could automatically generate optimal renting policies for the SaaS provider in the long run.

Despite having played a significant role in the Industry 4.0 era, the Internet of Things is currently faced with the challenge of how to ingest large-scale heterogeneous and multi-type device data. In response to this problem we present a heterogeneous device data ingestion model for an industrial big data platform. The model includes device templates and four strategies for data synchronization, data slicing, data splitting and data indexing, respectively. We can ingest device data from multiple sources with this heterogeneous device data ingestion model, which has been verified on our industrial big data platform. In addition, we present a case study on device data-based scenario analysis of industrial big data.

With the development and maturity of cloud storage, it has attracted a large number of users. Although cloud users do not need to concern themselves with the infrastructure used for storage, thus saving on equipment and maintenance costs, the sheer volume of data still generates significant cloud storage usage costs, which motivates cloud storage users to look for ways to further save costs. In this article, we analyze the whole process of using cloud storage to exhaustively explore opportunities, motivations, and challenges of cost optimization from user perspectives. Then we provide a comprehensive taxonomy and summary of recent advances in terms of storage efficiency (i.e., cost optimization by improving storage efficiency), cloud storage services (i.e., cost optimization by leveraging the features of cloud storage services), and emerging storage paradigms (i.e., cost optimization by leveraging emerging storage paradigms like edge storage). Finally, we present future directions for cost optimization from user perspectives and present our conclusion. This article offers a thorough survey of recent advances focusing on how to optimize the cost of using cloud storage for cloud users, and it has an opportunity to attract a broad audience in the cost-effective cloud storage market.

As increasingly complex functions are implemented in applications, directed acyclic graphs (DAGs) are widely used to model the inter-dependencies between individual functions. Cloud-based data processing platforms need to consider the complex topology of DAGs and arbitrary deadlines given by users for job scheduling, leading to an NP-hard decision-making problem. Leveraging spot instances in data processing platforms can achieve significant cost savings, but the unpredictable interruption of spot instances makes the problem of VM scaling and job scheduling more difficult. In this paper, a Reinforcement Learning (RL) based approach called SpotDAG is proposed to solve the auto-scaling problem for jobs modeled as DAGs on a data processing platform where spot instances are introduced. SpotDAG makes cluster scaling and job scheduling decisions at the same time by mapping its output to several meta-policies. This paper introduces the self-attention mechanism for feature extraction to help the intelligent agent learn faster. A mask layer after the output of the proposed RL-based algorithm circumvents illegal actions to ensure that a job is completed by its deadline. Extensive experimental results show that the proposed approach can significantly reduce the cost of instances for data processing platforms while ensuring that jobs are completed in time.

There has been a lot of research on flight delays. But it is more useful and difficult to estimate the departure delay time especially three hours before the scheduled time of departure, from which passengers can reasonably plan their travel time and the airline and airport staff can schedule flights more reasonably. In this paper, we develop a Spatio-temporal Graph Dual-Attention Neural Network (SGDAN) to learn the departure delay time for each flight with real-time conditions at three hours before the scheduled time of departure. Specifically, it first models the air traffic network as graph sequences, what is, using a heterogeneous graph to model a flight and its adjacent flights with the same departure or arrival airport in a special time interval, and using a sequence to model the flight and its previous flights that share the same aircraft. The main contributions of this paper are using heterogeneous graph-level attention to learn the influence between the flight and its adjacent flight together with sequence-level attention to learn the influence between the flight and its previous flight in the flight sequence. With aggregating features from the learned influence from both graph-level and sequence-level attention, SGDAN can generate node embedding to estimate the departure delay time. Experiments on a real-world large-scale data set show that SGDAN produces better results than state-of-the-art models in the accurate flight delay time estimation task.