11 July 2023 in IEEE Transactions on Parallel and Distributed Systems
Deep learning tasks (DLT) include training and inference tasks, where training DLTs have requirements on minimizing average job completion time (JCT) and inference tasks need sufficient GPUs to meet real-time performance. Unfortunately, existing work separately deploys multi-tenant training and inference GPU cluster, leading to the high JCT of training DLTs with limited GPUs when the inference cluster is under insufficient GPU utilization due to the periodic inference workload. DeepBoot solves the challenges by utilizing idle GPUs in the inference cluster for the training DLTs. Specifically, 1) DeepBoot designs adaptive task scaling (ATS) algorithm to allocate GPUs in the training and inference clusters for training DLTs and minimize the performance loss when reclaiming inference GPUs. 2) DeepBoot implements auto-fast elastic (AFE) training based on Pollux to reduce the restart overhead by inference GPU reclaiming. Our implementation on the testbed and large-scale simulation in Microsoft deep learning workload shows that DeepBoot can achieve 32% and 38% average JCT reduction respectively compared with the scheduler without utilizing idle GPUs in the inference cluster.
24 March 2024 in Proceedings of the AAAI Conference on Artificial Intelligence
Existing approaches defend against backdoor attacks in federated learning (FL) mainly through a) mitigating the impact of infected models, or b) excluding infected models. The former negatively impacts model accuracy, while the latter usually relies on globally clear boundaries between benign and infected model updates. However, in reality, model updates can easily become mixed and scattered throughout due to the diverse distributions of local data. This work focuses on excluding infected models in FL. Unlike previous perspectives from a global view, we propose Snowball, a novel anti-backdoor FL framework through bidirectional elections from an individual perspective inspired by one principle deduced by us and two principles in FL and deep learning. It is characterized by a) bottom-up election, where each candidate model update votes to several peer ones such that a few model updates are elected as selectees for aggregation; and b) top-down election, where selectees progressively enlarge themselves through picking up from the candidates. We compare Snowball with state-of-the-art defenses to backdoor attacks in FL on five real-world datasets, demonstrating its superior resistance to backdoor attacks and slight impact on the accuracy of the global model.
31 December 2021 in Automated Software Engineering
24 December 2024 in arXiv
Nowadays, research on GUI agents is a hot topic in the AI community. However, current research focuses on GUI task automation, limiting the scope of applications in various GUI scenarios. In this paper, we propose a formalized and comprehensive environment to evaluate the entire process of automated GUI Testing (GTArena), offering a fair, standardized environment for consistent operation of diverse multimodal large language models. We divide the testing process into three key subtasks: test intention generation, test task execution, and GUI defect detection, and construct a benchmark dataset based on these to conduct a comprehensive evaluation. It evaluates the performance of different models using three data types: real mobile applications, mobile applications with artificially injected defects, and synthetic data, thoroughly assessing their capabilities in this relevant task. Additionally, we propose a method that helps researchers explore the correlation between the performance of multimodal language large models in specific scenarios and their general capabilities in standard benchmark tests. Experimental results indicate that even the most advanced models struggle to perform well across all sub-tasks of automated GUI Testing, highlighting a significant gap between the current capabilities of Autonomous GUI Testing and its practical, real-world applicability. This gap provides guidance for the future direction of GUI Agent development. Our code is available at https://github.com/ZJU-ACES-ISE/ChatUITest.