Mooncake is the serving platform for Kimi, an LLM chatbot service developed by Moonshot AI. This platform features a KVCache-centric disaggregated architecture that not only separates prefill and decoding clusters but also efficiently utilizes the underexploited CPU, DRAM, SSD and NIC resources of the GPU cluster to establish a disaggregated KVCache. At the core of Mooncake is its KVCache-centric global cache and a scheduler designed to maximize throughput while adhering to stringent latency-related Service Level Objectives (SLOs). Our experiments demonstrate that Mooncake excels in scenarios involving long-context inputs. In tests using real traces, Mooncake increases the effective request capacity by 59% ∼ 498% when compared to baseline methods, all while complying with SLOs. Currently, Mooncake is operational across thousands of nodes, processing over 100 billion tokens daily. In practical deployments, Mooncake ’s innovative architecture enables Kimi to handle 115% and 107% more requests on NVIDIA A800 and H800 clusters, respectively, compared to previous systems.

To secure cloud storage and enforce access control, data encryption has become essential, given the ever increasing cyber threat everywhere. Attribute-based Encryption (ABE) crypto systems are widely considered as a promising solution under such a context for its security strength, scalability and control flexibility. One major challenge, however, for applying ABE-based techniques in real world applications is its high overhead in various aspects. In this research, we are particularly concerned with the storage size expansion in existing ABE schemes. This combined with the vast-size nature of the cloud data poses an enormous challenge to the effective usage of the cloud data storage space and affects the utility of data deduplication. Normally, data deduplication is carried out based on identifying similar and even identical contents both within and between data files, however, these patterns will be destroyed after performing data encryption using any semantically secure encryption scheme including ABE. In this research, we focus on ciphertexts deduplication under ABE, which to our best knowledge is the first of such an effort. Our fundamental observation stems from the structure of ABE ciphertexts and the possible similarities among different access structures. We show how to design a secure ciphertext deduplication scheme based on a classical CP-ABE scheme by innovatively modifying the construction with a recursive algorithm, eliminating the duplicated secrets and adding additional randomness to some certain ciphertext. We then give a detailed analysis on the proposed scheme with respect to both efficiency and security. To thoroughly assess the performance of the proposed scheme, we also implement a prototype system and conduct comprehensive experiments, which shows that our ciphertext reduplication scheme could reduce up to 80% computation and storage cost in the best case.

The densely deployed Access Points (APs) have overlapping coverage areas. In the hotspot area, a user can usually receive signals of more than ten APs. The signal-based association in IEEE 802.11 may result in significant unbalanced loads among APs. Moreover, diverse user demands on bandwidth further exacerbate the load unbalance. Some APs are too overloaded when multiple high-demand users gather on them due to the strongest signal, while the others are light-loaded with a few low-demand users associated to them. The severe load unbalance degrades the user performance. In this paper, by adding different user bandwidth demands as new constraints, we formulate the joint AP association and bandwidth allocation problem. We comprehensively analyze the solution space of optimal bandwidth allocation while satisfying the time-based fairness among users. We develop a 1/2-approximation algorithm to solve the problem. Our extensive trace- driven evaluations show that our algorithm achieves better load balance. As a result, it greatly improves the aggregated throughput and provides better user fairness than conventional association schemes.