Tolga Kerimoglu

Tolga Kerimoglu

Auto-generated profile, based on publicly available information

Tolga Kerimoglu

ETH Zurich, Zurich, Switzerland

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 April 2024
Verified
Conference Paper
Open access
FSFoteini Strati
Foteini Strati
PEPaul Elvinger
Paul Elvinger
TKTolga Kerimoglu
Tolga Kerimoglu
AKAna Klimovic
Ana Klimovic
The widespread adoption of ML has led to a high demand for GPU hardware and consequently, severe shortages of GPUs in the public cloud. Allocating a sufficient number of GPUs to train or fine-tune today's large ML models in a single cloud region is often difficult. Users can get access to more GPUs if they are willing to run a ML training job using devices across different geographical regions. However, GPU nodes are connected with lower network bandwidth and cloud providers charge extra for data transfers across geographical regions. In this work, we explore when and how it makes sense to leverage GPUs across zones and regions for distributed ML training. We analyze the throughput and cost impact of cross-region training based on the computation and communication patterns of different model parallelism strategies, develop a profile-based analytical model for estimating training throughput and cost, and provide guidelines for allocating geo-distributed resources efficiently. We find that although ML training throughput and cost with pure data parallelism degrades significantly when nodes span geographic regions, cross-region training with pipeline parallelism is practical.

2 April 2026 in Proceedings of the ACM on Management of Data
Verified
Journal Article
Open access
MBMaximilian Böther
Maximilian Böther
XYXiaozhe Yao
Xiaozhe Yao
TKTolga Kerimoglu
Tolga Kerimoglu
DGDan Graur
Dan Graur
VGViktor Gsteiger
Viktor Gsteiger
AKAna Klimovic
Ana Klimovic
State-of-the-art large language and vision models are trained over trillions of tokens that are aggregated from a large variety of sources. As training data collections grow, manually managing the samples becomes time-consuming, tedious, and prone to errors. Yet recent research shows that the data mixture and the order in which samples are visited during training can significantly influence model accuracy. We build and present Mixtera , a data plane for foundation model training that enables users to declaratively express which data samples should be used in which proportion and in which order during training. Mixtera is a centralized, read-only layer that is deployed on top of existing training data collections and can be declaratively queried. It operates independently of the filesystem structure and supports mixtures across arbitrary properties (e.g., language, source dataset) as well as dynamic adjustment of the mixture based on model feedback. We experimentally evaluate Mixtera and show that our implementation does not bottleneck training and scales to 256 GH200 superchips. We demonstrate how Mixtera supports recent advancements in mixing strategies by implementing the Adaptive Data Optimization (ADO) algorithm in the system and evaluating its performance impact. We also show how Mixtera enables exploring the role of mixtures for vision-language models, which is a growing area of research.