Mehran Salmani

Mehran Salmani

Auto-generated profile, based on publicly available information

Mehran Salmani

Computer Science and Automation, TU Ilmenau, Ilmenau, Germany

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
8 May 2023
Verified
Conference Paper
Open access
MSMehran Salmani
Mehran Salmani
SGSaeid Ghafouri
Saeid Ghafouri
ASAlireza Sanaee
Alireza Sanaee
KRKamran Razavi
Kamran Razavi
MMMax Mühlhäuser
Max Mühlhäuser
MSMohsen Sharifi
Mohsen Sharifi
The use of machine learning (ML) inference for various applications is growing drastically. ML inference services engage with users directly, requiring fast and accurate responses. Moreover, these services face dynamic workloads of requests, imposing changes in their computing resources. Failing to right-size computing resources results in either latency service level objectives (SLOs) violations or wasted computing resources. Adapting to dynamic workloads considering all the pillars of accuracy, latency, and resource cost is challenging. In response to these challenges, we propose InfAdapter, which proactively selects a set of ML model variants with their resource allocations to meet latency SLO while maximizing an objective function composed of accuracy and cost. InfAdapter decreases SLO violation and costs up to 65% and 33%, respectively, compared to a popular industry autoscaler (Kubernetes Vertical Pod Autoscaler).

5 May 2025
Verified
Conference Paper
Open access
SGSaeid Ghafouri
Saeid Ghafouri
KRKamran Razavi
Kamran Razavi
MSMehran Salmani
Mehran Salmani
ASAlireza Sanaee
Alireza Sanaee
TBTania Lorido Botran
Tania Lorido Botran
PJPooyan Jamshidi
Pooyan Jamshidi

27 April 2026
Verified
Conference Paper
Open access
MSMehran Salmani
Mehran Salmani
KRKamran Razavi
Kamran Razavi
PAPeter Amthor
Peter Amthor
BKBoris Koldehofe
Boris Koldehofe
Autoscaling mechanisms have been used by inference serving systems to maintain service quality and efficiently use cloud resources (CPUs/GPUs). Horizontal scaling-based mechanisms have a responsiveness issue due to cold-start delays. In-place CPU vertical scaling adopted by recent works avoids cold-start; however, it is constrained by hardware limits and model parallelizability, and is limited to CPUs, which are efficient for low workloads but cost-inefficient for large workloads. Thus, combining CPUs and GPUs under dynamic workloads is worthwhile. However, enabling responsive switching between them is challenging. We propose VCG, an inference serving system that utilizes different resources based on the workload, and enables a smooth transition from CPUs to GPUs and vertically scaling GPUs (adjusting the GPU percentage) by forking a new process with enough GPU compute resources using CUDA MPS, and delivering inference requests to the child process through inter-process communication pipes. When the capacity of service is (nearly) exhausted, VCG uses horizontal scaling to provide scalability headroom for future steps. We evaluate a prototype of VCG under two different workloads and show that it reduces SLO violations up to 8× compared to a baseline using horizontal scaling of GPU instances under a bursty workload. It can also reduce the cost by around 60% compared to two CPU-based autoscaling baselines under a dynamic (non-bursty) workload.