5 January 2018 in IEEE Transactions on Parallel and Distributed Systems
With increasing GPU-intensive workloads deployed on cloud, cloud service providers are seeking for practical and efficient GPU virtualization solutions. However, the cutting-edge GPU virtualization techniques such as gVirt still suffer from the restriction of scalability, which constrains the number of guest virtual GPU instances. This paper presents gScale, a scalable and practical open source GPU virtualization solution based on gVirt. gScale presents a sharing mechanism which combines partition and sharing together to break the hardware limitation of global graphics memory space. Particularly, we propose two approaches for gScale: (1) the private shadow graphics translation table (GTT), which enables global graphics memory space sharing among virtual GPUs, (2) ladder mapping and fence memory space pool, which allows CPU access host physical memory space (serving the graphics memory) to bypass global graphics memory space. To mitigate the performance degradation when the number of vGPUs scales up, three other mechanisms are proposed: (1) slot sharing, which improves the performance of vGPU by dividing the high global graphics memory into multiple slots, (2) predictive GTT copy mechanism, which reduces the performance loss by switching private shadow GTT before context switch, (3) predictive-copy aware scheduling, which maximizes the improvement of predictive GTT copy mechanism.
1 June 2021 in Journal of Systems Architecture
16 October 2019 in IEEE Transactions on Parallel and Distributed Systems
This paper introduces gMig, an open-source and practical vGPU live migration solution for full virtualization. Taking the advantage of the dirty pattern of GPU workloads, gMig presents the One-Shot Pre-Copy mechanism combined with the hashing based Software Dirty Page technique to achieve efficient vGPU live migration. Particularly, we propose three core techniques for gMig: 1) Dynamic Graphics Address Remapping, which parses and manipulates GPU commands to adjust the address mapping and adapt to a different environment after migration, 2) Software Dirty Page, which utilizes a hashing based approach with sampling pre-filtering to detect page modification, overcomes the commodity GPU's hardware limitation, and speeds up the migration by only sending the dirtied pages, 3) Overlapped Migration Process, which significantly compresses the hanging overhead by overlapping the dirty page verification and transmission concurrently. Our evaluation shows that gMig achieves GPU live migration with an average downtime of 302 ms on Windows and 119 ms on Linux. With the help of Software Dirty Page, the number of GPU pages transferred during the downtime is effectively reduced by up to 80.0 percent . The design of sampling filter and overlapped processing can bring about further 30.0 and 10.0 percent improvements in page processing.