Lukas Gianinazzi

Lukas Gianinazzi

Auto-generated profile, based on publicly available information

Lukas Gianinazzi

ETH Zurich, Zurich, Switzerland

Research Focus & Metrics

Academic Career Length

7Years (2020-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
3 June 2024
Verified
Conference Paper
Open access
PLPiotr Luczynski
Piotr Luczynski
LGLukas Gianinazzi
Lukas Gianinazzi
PIPatrick Iff
Patrick Iff
LWLeighton Wilson
Leighton Wilson
DSDaniele De Sensi
Daniele De Sensi
THTorsten Hoefler
Torsten Hoefler
Efficient Reduce and AllReduce communication collectives are a critical cornerstone of high-performance computing (HPC) applications. We present the first systematic investigation of Reduce and AllReduce on the Cerebras Wafer-Scale Engine (WSE). This architecture has been shown to achieve unprecedented performance both for machine learning workloads and other computational problems like FFT. We introduce a performance model to estimate the execution time of algorithms on the WSE and validate our predictions experimentally for a wide range of input sizes. In addition to existing implementations, we design and implement several new algorithms specifically tailored to the architecture. Moreover, we establish a lower bound for the runtime of a Reduce operation on the WSE. Based on our model, we automatically generate code that achieves near-optimal performance across the whole range of input sizes. Experiments demonstrate that our new Reduce and AllReduce algorithms outperform the current vendor solution by up to 3.27×. Additionally, our model predicts performance with less than 4% error. The proposed communication collectives increase the range of HPC applications that can benefit from the high throughput of the WSE. Our model-driven methodology demonstrates a disciplined approach that can lead the way to further algorithmic advancements on wafer-scale architectures.

26 April 2026
Verified
Conference Paper
Open access
TBTommaso Bonato
Tommaso Bonato
AKAbdul Kabbani
Abdul Kabbani
AGAhmad Ghalayini
Ahmad Ghalayini
MPMichael Papamichael
Michael Papamichael
MDMohammad Dohadwala
Mohammad Dohadwala
THTorsten Hoefler
Torsten Hoefler

20 February 2024
Verified
Conference Paper
Open access
LGLukas Gianinazzi
Lukas Gianinazzi
AZAlexandros Nikolaos Ziogas
Alexandros Nikolaos Ziogas
LHLangwen Huang
Langwen Huang
PLPiotr Luczynski
Piotr Luczynski
SASaleh Ashkboosh
Saleh Ashkboosh
THTorsten Hoefler
Torsten Hoefler
We propose a novel approach to iterated sparse matrix dense matrix multiplication, a fundamental computational kernel in scientific computing and graph neural network training. In cases where matrix sizes exceed the memory of a single compute node, data transfer becomes a bottleneck. An approach based on dense matrix multiplication algorithms leads to sub-optimal scalability and fails to exploit the sparsity in the problem. To address these challenges, we propose decomposing the sparse matrix into a small number of highly structured matrices called arrow matrices, which are connected by permutations. Our approach enables communication-avoiding multiplications, achieving a polynomial reduction in communication volume per iteration for matrices corresponding to planar graphs and other minor-excluded families of graphs. Our evaluation demonstrates that our approach outperforms a state-of-the-art method for sparse matrix multiplication on matrices with hundreds of millions of rows, offering near-linear strong and weak scaling.

11 November 2023
Verified
Conference Paper
Open access
MBMaciej Besta
Maciej Besta
PRPawel Renc
Pawel Renc
RGRobert Gerstenberger
Robert Gerstenberger
PLPaolo Sylos Labini
Paolo Sylos Labini
AZAlexandros Ziogas
Alexandros Ziogas
THTorsten Hoefler
Torsten Hoefler
Graph attention models (A-GNNs), a type of Graph Neural Networks (GNNs), have been shown to be more powerful than simpler convolutional GNNs (C-GNNs). However, A-GNNs are more complex to program and difficult to scale. To address this, we develop a novel mathematical formulation, based on tensors that group all the feature vectors, targeting both training and inference of A-GNNs. The formulation enables straightforward adoption of communication-minimizing routines, it fosters optimizations such as vectorization, and it enables seamless integration with established linear algebra DSLs or libraries such as GraphBLAS. Our implementation uses a data redistribution scheme explicitly developed for sparse-dense tensor operations used heavily in GNNs, and fusing optimizations that further minimize memory usage and communication cost. We ensure theoretical asymptotic reductions in communicated data compared to the established message-passing GNN paradigm. Finally, we provide excellent scalability and speedups of even 4--5x over modern libraries such as Deep Graph Library.

7 August 2023
Verified
Conference Paper
Open access
TMTiziano De Matteis
Tiziano De Matteis
LGLukas Gianinazzi
Lukas Gianinazzi
JLJohannes de Fine Licht
Johannes de Fine Licht
THTorsten Hoefler
Torsten Hoefler
Dataflow devices represent an avenue towards saving the control and data movement overhead of Load-Store Architectures. Various dataflow accelerators have been proposed, but how to efficiently schedule applications on such devices remains an open problem. The programmer can explicitly implement both temporal and spatial parallelism, and pipelining across multiple processing elements can be crucial to take advantage of the fast on-chip interconnect, enabling the concurrent execution of different program components. This paper introduces canonical task graphs, a model that enables streaming scheduling of task graphs over dataflow architectures. We show how a task graph can be statically analyzed to understand its steady-state behavior, and we use this information to partition it into temporally multiplexed components of spatially executed tasks. Results on synthetic and realistic workloads show how streaming scheduling can increase speedup and device utilization over a traditional scheduling approach.

30 June 2021 in ACM Transactions on Parallel Computing
Verified
Journal Article
Open access
BGBarbara Geissmann
Barbara Geissmann
LGLukas Gianinazzi
Lukas Gianinazzi
We present the first near-linear work and poly-logarithmic depth algorithm for computing a minimum cut in an undirected graph. Previous parallel algorithms with poly-logarithmic depth required at least quadratic work in the number of vertices. In a graph with n vertices and m edges, our randomized algorithm computes the minimum cut with high probability in O ( m log 4 n ) work and O (log 3 n ) depth. This result is obtained by parallelizing a data structure that aggregates weights along paths in a tree, in addition exploiting the connection between minimum cuts and approximate maximum packings of spanning trees. In addition, our algorithm improves upon bounds on the number of cache misses incurred to compute a minimum cut.