The Internet of Things (IoT)-enabled applications use sensors and actuators to collect big data, which are processed by big data models, e.g., Spark. Generally, data processing tasks are precedence constrained and the computation results are transmitted to other IoT devices. In this article, we consider the Spark workflow problem of scheduling tasks with data affinity to heterogeneous servers to minimize the maximum completion time. In a Spark instance, jobs are precedence constrained and stages for each job are also precedence constrained. There are a large number of topological stage orders. A balance between task execution times, determined by heterogeneous servers, and transmission times caused by data affinity is difficult to achieve. A scheduling optimization algorithm framework is proposed, which consists of five components: 1) temporal parameter calculation; 2) ready stage adding; 3) task sequencing; 4) resource allocation; and 5) schedule improvement. Strategies for each component are developed. The algorithmic components are statistically calibrated over a comprehensive set of instances. The proposed algorithm is compared to two modified classic algorithms for similar problems on typical scientific workflow instances. The experimental results demonstrate the effectiveness of the proposal for the considered problem.

In the credit cloud, credit services are sold to applications for credit computing, credit fusion and credit risk estimates. Plenty of services with different performance for the same task may have different execution time and charged by various ways. The users have specific requirements for the workflow completion time or cost. Hence, to meet the user’s satisfaction is an important challenge. In this paper, we propose heuristic scheduling methods for credit workflow with total cost minimization, and the deadline should be satisfied. The problem can be divided into two sub-problems, task-mode mapping and task tabling on renting service instances. For the task-mode mapping problem, a recursive heuristic method is constructed to select appropriate service for each task of the workflow. Then another heuristic algorithm based is established to get a final schema with deadline constraint. We discussed the service instance rented in shareable manner and compared with un-shareable manner. Three renting strategies are discussed in detail. Experimental results show the effectiveness and efficiency of the proposed algorithm.