Workflows are among the most commonly used tools in a variety of execution environments. Many of them target a specific environment; few of them make it possible to execute an entire workflow in different environments, e.g. Kubernetes and batch clusters. We present a novel approach to workflow execution, called StreamFlow, that complements the workflow graph with the declarative description of potentially complex execution environments, and that makes it possible the execution onto multiple sites not sharing a common data space. StreamFlow is then exemplified on a novel bioinformatics pipeline for single cell transcriptomic data analysis workflow.
1 March 2022 in Future Generation Computer Systems
The performance bottleneck in file-based workflows remains a pressing issue in the realm of I/O-based workflows. To address this challenge, a novel annotation language has been developed. CAPIO-CL is positioned as an innovative I/O coordination language, enabling users to annotate data dependencies within file-based workflows with synchronization semantics pertinent to the involved files and directories. Through the information provided by the language, optimization opportunities arise in streaming and preemptive data movement. This paper serves to illustrate the semantics and syntax enabling CAPIO-CL to enhance the performance of in situ workflows without necessitating the rewriting or modification of the original workflow application steps. Finally, an analysis of CAPIO-CL is provided, taking into consideration both language expressiveness and application performance enhancement.
27 September 2025 in Future Generation Computer Systems
The Grid is currently living an exciting evolution step to turn from a niche technological infrastructure to a widely used Business environment. To this aim, Workflows are playing an important role acting as the link towards the well-established Business Process Management environment. A number of Grid projects worldwide are trying to implement this link and create the Business Grid but no definitive proposal has been provided yet. In this paper, we analyse two European Grid projects, Akogrimo and NextGRID, both investigating the Business usage of Grid from architectural perspectives but each one adopting a different approach in proposing solutions. The comparative analysis we report provides interesting elements to define a unified vision of the Next Generation Grids where Workflow Models, Languages, and Engines constitute a fundamental building block. Understanding and sharing this vision can be the first step to get to an effective architectural design for Business Grids.