Ben Blamey

Ben Blamey

orcid

Research Focus & Metrics

Academic Career Length

15Years (2012-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications


1 July 2018
Verified
Conference Paper
Closed access
PTPreechakorn Torruangwatthana
Preechakorn Torruangwatthana
HWHakan Wieslander
Hakan Wieslander
BBBen Blamey
Ben Blamey
AHAndreas Hellander
Andreas Hellander
STSalman Toor
Salman Toor
Many streaming frameworks have been introduced to deal with the needs for online analysis of massive datasets. Scientific applications often require significant changes to make them compatible with these frameworks. Other issues include tight coupling with the underlying infrastructure, shared computing environment, static topology settings, and complex configuration. In this article we present HarmonicIO, a lightweight streaming framework specialized for scientific datasets. It boasts a smart dynamic architecture, is highly elastic, and enforces a clear separation between framework components and application execution environment using container technology.

1 May 2019
Verified
Conference Paper
Closed access
BBBen Blamey
Ben Blamey
FWFredrik Wrede
Fredrik Wrede
JKJohan Karlsson
Johan Karlsson
AHAndreas Hellander
Andreas Hellander
STSalman Toor
Salman Toor
Top-K queries are an established heuristic in information retrieval. This paper presents an approach for optimal tiered storage allocation under stream processing workloads using this heuristic: those requiring the analysis of only the top-K ranked most relevant documents from a fixed-length stream, stream window, or batch job. Documents are ranked for relevance on a user-specified interestingness function, the top-K stored for further processing. This scenario bears similarity to the classic Secretary Hiring Problem (SHP), and the expected rate of document writes and document lifetime can be modelled as a function of document index. We present parameter-based algorithms for storage tier placement, minimizing document storage and transport costs. We derive expressions for optimal parameter values in terms of tier storage and transport costs a priori, without needing to monitor the application. This contrasts with (often complex) existing work on tiered storage optimization, which is either tightly coupled to specific use cases, or requires active monitoring of application IO load - ill-suited to long-running or one-off operations common in the scientific computing domain. We motivate and evaluate our model with a trace-driven simulation of human-in-the-loop bio-chemical model exploration, and two cloud storage case studies.

7 November 2013
Verified
Book Chapter
Closed access
PDF file
BBBen Blamey
Ben Blamey
TCTom Crick
Tom Crick
GOGiles Oatley
Giles Oatley

19 March 2021 in GigaScience
Verified
Journal Article
Open access
PDF file
BBBen Blamey
Ben Blamey
STSalman Toor
Salman Toor
MDMartin Dahlö
Martin Dahlö
HWHåkan Wieslander
Håkan Wieslander
PHPhilip J Harrison
Philip J Harrison
AHAndreas Hellander
Andreas Hellander
Background: Large streamed datasets, characteristic of life science applications, are often resource-intensive to process, transport and store. We propose a pipeline model, a design pattern for scientific pipelines, where an incoming stream of scientific data is organized into a tiered or ordered “data hierarchy". We introduce the HASTE Toolkit, a proof-of-concept cloud-native software toolkit based on this pipeline model, to partition and prioritize data streams to optimize use of limited computing resources. Findings: In our pipeline model, an “interestingness function” assigns an interestingness score to data objects in the stream, inducing a data hierarchy. From this score, a “policy” guides decisions on how to prioritize computational resource use for a given object. The HASTE Toolkit is a collection of tools to adopt this approach. We evaluate with 2 microscopy imaging case studies. The first is a high content screening experiment, where images are analyzed in an on-premise container cloud to prioritize storage and subsequent computation. The second considers edge processing of images for upload into the public cloud for real-time control of a transmission electron microscope. Conclusions: Through our evaluation, we created smart data pipelines capable of effective use of storage, compute, and network resources, enabling more efficient data-intensive experiments. We note a beneficial separation between scientific concerns of data priority, and the implementation of this behaviour for different resources in different deployment contexts. The toolkit allows intelligent prioritization to be `bolted on' to new and existing systems – and is intended for use with a range of technologies in different deployment scenarios.