William S. Moses

William S. Moses

Auto-generated profile, based on publicly available information

William S. Moses

Siebel School of Computing and Data Science, University of Illinois Urbana-Chamapign, IL, USA

Research Focus & Metrics

Academic Career Length

11Years (2016-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 March 2025
Verified
Conference Paper
Open access
MLMartin Paul Lücke
Martin Paul Lücke
OZOleksandr Zinenko
Oleksandr Zinenko
WMWilliam S. Moses
William S. Moses
MSMichel Steuwer
Michel Steuwer
ACAlbert Cohen
Albert Cohen
To take full advantage of a specific hardware target, performance engineers need to gain control on compilers in order to leverage their domain knowledge about the program and hardware. Yet, modern compilers are poorly controlled, usually by configuring a sequence of coarse-grained monolithic black-box passes, or by means of predefined compiler annotations/pragmas. These can be effective, but often do not let users precisely optimize their varying compute loads. As a consequence, performance engineers have to resort to implementing custom passes for a specific optimization heuristic, requiring compiler engineering expert knowledge. In this paper, we present a technique that provides fine-grained control of general-purpose compilers by introducing the Transform dialect, a controllable IR-based transformation system implemented in MLIR. The Transform dialect empowers performance engineers to optimize their various compute loads by composing and reusing existing---but currently hidden---compiler features without the need to implement new passes or even rebuilding the compiler. We demonstrate in five case studies that the Transform dialect enables precise, safe composition of compiler transformations and allows for straightforward integration with state-of-the-art search methods.

29 June 2018 in arXiv
Verified
Preprint
Open access
PDF file
NVNicolas Vasilache
Nicolas Vasilache
OZOleksandr Zinenko
Oleksandr Zinenko
TTTheodoros Theodoridis
Theodoros Theodoridis
PGPriya Goyal
Priya Goyal
ZDZachary DeVito
Zachary DeVito
ACAlbert Cohen
Albert Cohen
Deep learning models with convolutional and recurrent networks are now ubiquitous and analyze massive amounts of audio, image, video, text and graph data, with applications in automatic translation, speech-to-text, scene understanding, ranking user preferences, ad placement, etc. Competing frameworks for building these networks such as TensorFlow, Chainer, CNTK, Torch/PyTorch, Caffe1/2, MXNet and Theano, explore different tradeoffs between usability and expressiveness, research or production orientation and supported hardware. They operate on a DAG of computational operators, wrapping high-performance libraries such as CUDNN (for NVIDIA GPUs) or NNPACK (for various CPUs), and automate memory allocation, synchronization, distribution. Custom operators are needed where the computation does not fit existing high-performance library calls, usually at a high engineering cost. This is frequently required when new operators are invented by researchers: such operators suffer a severe performance penalty, which limits the pace of innovation. Furthermore, even if there is an existing runtime call these frameworks can use, it often doesn't offer optimal performance for a user's particular network architecture and dataset, missing optimizations between operators as well as optimizations that can be done knowing the size and shape of data. Our contributions include (1) a language close to the mathematics of deep learning called Tensor Comprehensions, (2) a polyhedral Just-In-Time compiler to convert a mathematical description of a deep learning DAG into a CUDA kernel with delegated memory management and synchronization, also providing optimizations such as operator fusion and specialization for specific sizes, (3) a compilation cache populated by an autotuner. [Abstract cutoff]


28 January 2026 in The Astrophysical Journal
Verified
Journal Article
Open access
PTPaul Tiede
Paul Tiede
WMWilliam Moses
William Moses
VCValentin Churavy
Valentin Churavy
MJMichael Johnson
Michael Johnson
DPDominic W Pesce
Dominic W Pesce
PGPeter Galison
Peter Galison
Very long baseline interferometry (VLBI) achieves the highest angular resolution in astronomy. VLBI measures corrupted Fourier components, known as visibilities. Reconstructing on-sky images from these visibilities is a challenging inverse problem, particularly for sparse arrays such as the Event Horizon Telescope (EHT) and the Very Long Baseline Array, where incomplete sampling and severe calibration errors introduce significant uncertainty in the image. To help guide convergence and control the uncertainty in image reconstructions, regularization on the space of images is utilized, such as enforcing smoothness or similarity to a fiducial image. Coupled with this regularization is the introduction of a new set of parameters that modulate its strength. We present a hierarchical Bayesian imaging approach (hierarchical interferometric Bayesian Imaging, HIBI) that enables the quantification of uncertainty for all parameters. Incorporating instrumental effects within HIBI is straightforward, allowing for simultaneous imaging and calibration of data. To showcase HIBI’s effectiveness and flexibility, we build a simple imaging model based on Markov random fields and demonstrate how different physical components can be included, e.g., black hole shadow size, and their uncertainties can be inferred. For example, while the original EHT publications were unable to constrain the ring width of M87*, HIBI measures a width of 9.3 ± 1.3 μ as. We apply HIBI to image and calibrate EHT synthetic data, real EHT observations of M87*, and multifrequency observations of OJ 287. Across these tests, HIBI accurately recovers a wide variety of image structures and quantifies their uncertainties. HIBI is publicly available in the Comrade VLBI software repository.

9 October 2025 in Proceedings of the ACM on Programming Languages
Verified
Journal Article
Open access
AVArya Vohra
Arya Vohra
LLLeo Seojun Lee
Leo Seojun Lee
JBJakub Bachurski
Jakub Bachurski
OZOleksandr Zinenko
Oleksandr Zinenko
PPPhitchaya Mangpo Phothilimthana
Phitchaya Mangpo Phothilimthana
WMWilliam S. Moses
William S. Moses
Machine learning (ML) compilers rely on graph-level transformations to enhance the runtime performance of ML models. However, performing local transformations on individual operations can create effects far beyond the location of the rewrite. In particular, a local rewrite can change the profitability or legality of hard-to-predict downstream transformations, particularly regarding data layout, parallelization, fine-grained scheduling, and memory management. As a result, program transformations are often driven by manually-tuned compiler heuristics, which are quickly rendered obsolete by new hardware and model architectures. Instead of hand-written local heuristics, we propose the use of equality saturation. We replace such heuristics with a more robust global performance model, which accounts for downstream transformations. Equality saturation addresses the challenge of local optimizations inadvertently constraining or negating the benefits of subsequent transformations, thereby providing a solution that is inherently adaptable to newer workloads. While this approach still requires a global performance model to evaluate the profitability of transformations, it holds significant promise for increased automation and adaptability. This paper addresses challenges in applying equality saturation on real-world ML compute graphs and state-of-the-art hardware. By doing so, we present an improved method for discovering effective compositions of graph optimizations. We study different cost modeling approaches to deal with fusion and layout optimization, and tackle scalability issues that arise from considering a very wide range of algebraic optimizations. We design an equality saturation pass for the XLA compiler, with an implementation in C++ and Rust. We demonstrate an average speedup of 3.45% over XLA’s optimization flow across our benchmark suite on various CPU and GPU platforms, with a maximum speedup of 56.26% for NasRNN on CPU.

15 November 2025
Verified
Conference Paper
Open access
FHFaveo Hoerold
Faveo Hoerold
IIIvan R. Ivanov
Ivan R. Ivanov
ADAkash Dhruv
Akash Dhruv
WMWilliam S. Moses
William S. Moses
ADAnshu Dubey
Anshu Dubey
JDJens Domke
Jens Domke