Spark has become one of the main options for large-scale analytics running on top of shared-nothing clusters. This work aims to make a deep dive into the parallelism configuration and shed light on the behavior of parallel spark jobs. It is motivated by the fact that running a Spark application on all the available processors does not necessarily imply lower running time, while may entail waste of resources. We first propose analytical models for expressing the running time as a function of the number of machines employed. We then take another step, namely to present novel algorithms for configuring dynamic partitioning with a view to minimizing resource consumption without sacrificing running time beyond a user-defined limit. The problem we target is NP-hard. To tackle it, we propose a greedy approach after introducing the notions of dependency graphs and of the benefit from modifying the degree of partitioning at a stage; complementarily, we investigate a randomized approach. Our polynomial solutions are capable of judiciously use the resources that are potentially at user's disposal and strike interesting trade-offs between running time and resource consumption. Their efficiency is thoroughly investigated through experiments based on real execution data.

Business process redesign (BPR) is an organizational initiative for achieving competitive multi-faceted advantages regarding business processes, in terms of cycle time, quality, cost, customer satisfaction and other critical performance metrics. In spite of the fact that BPR tools and methodologies are increasingly being adopted, process innovation efforts have proven ineffective in delivering the expected outcome. This paper investigates the eligibility of BPMN process models towards the application of redesign methods inspired by data-flow communities. In previous work, the transformation of a business process model to a directed acyclic graph (DAG) has yielded notable optimization results for determining average performance of process executions consisting of ad-hoc processes. Still, the utilization encountered drawbacks due to a lack of input specification, complexity assessment and normalization of the BPMN model and application to more generic business process cases. This paper presents an assessment mechanism that measures the eligibility of a BPMN model and its capability to be effectively transformed to a DAG and be further subjected to data-centric workflow optimization methods. The proposed mechanism evaluates the model type, complexity metrics, normalization and optimization capability of candidate process models, while at the same time allowing users to set their desired complexity thresholds. An indicative example is used to demonstrate the assessment phases and to illustrate the usability of the proposed mechanism towards the advancement and facilitation of the optimization phase. Finally, the authors review BPMN models from both an SOA-based business process design (BPD) repository and relevant literature and assess their eligibility.

We deal with the problem of predictive maintenance (PdM) in a vehicle fleet management setting following an unsupervised streaming anomaly detection approach. We investigate a variety of unsupervised methods for anomaly detection, such as proximity-based, hybrid (statistical and proximity-based) and transformers. The proposed methods can properly model the context in which each member of the fleet operates. In our case, the context is both crucial for effective anomaly detection and volatile, which calls for streaming solutions that take into account only the recent values. We propose two novel techniques, a 2-stage proximity-based one and context-aware transformers along with advanced thresholding. In addition, to allow for testing PdM techniques for vehicle fleets in a fair and reproducible manner, we build a new fleet-like benchmarking dataset based on an existing dataset of turbofan simulations. Our evaluation results show that our proposals reduce the maintenance costs compared to existing solutions.