Emerging cloud-native distributed databases rely on local NVMe SSDs to provide high-performance and highavailable data services to many cloud applications. However, the database clusters suffer from low utilization of local storage because of the imbalance between CPU and storage capacities within each node. For instance, the OceanBase distributed database cluster, with hundreds of PB local storage capacity, only utilizes around 40% of its local storage. Although disaggregated storage (EBS) can enhance storage utilization by provisioning the CPU and storage independently on demand, they suffer from performance bottlenecks and high costs. In this paper, we propose LightPool, a high-performance and lightweight storage pool architecture large-scale deployed in the OceanBase clusters, enhancing storage resource utilization. The key idea of LightPool is aggregating cluster storage into a storage pool and enabling unified management. In particular, LightPool adopts NVMe-oF to enable high-performance storage resource sharing among cluster nodes and integrate the storage pool with Kubernetes to achieve flexible management and allocation of storage resources. Furthermore, we design the hot-upgrade and hot-migration mechanisms to enhance the availability of LightPool. We have deployed LightPool on over 8500 nodes in production clusters. Statistics show that LightPool can improve storage resource utilization from about 40% to 65%. Experimental results show that the extra latency from LightPool is only about 2.1 μs compared to local storage. Compared to OpenEBS, LightPool enhances bandwidth up to 190.9% in microbenchmarks and throughput up to 6.9% in real-world applications. LightPool is the best practice to deploy NVMe-oF (NVMe/TCP) in the production environment. We also discuss important lessons and experiences learned from the development of LightPool.

The NVMe-over-Fabrics (NVMe-oF) is gaining popularity in cloud data centers as a remote storage protocol for accessing NVMe storage devices across servers. With the rapid increase in throughput of the NVMe storage devices, the NVMe-oF stack consumes a significant amount of valuable CPU resources. To release these valuable computing power to other tasks, many smartNICs now support NVMe-oF Target offloading. However, this emerging NVMe-oF offloading scheme's performance has not been fully investigated. In this work, we conducted comprehensive evaluations on smartNIC NVMe-oF target offloading and non-offloading NVMe-oF with TCP and RDMA. Our results demonstrate that the target offloading can achieve comparable throughput compared to NVMe/RDMA in most synthetic and real-world application workloads while releasing up to 38.7% CPU resources. Additionally, target offloading benefits from host bypass and peer-to-peer communication between smartNIC and NVMe SSD, resulting in only 68.9% to 95.8% average latency of NVMe/RDMA. However, we also found that target offloading schemes lack flexibility for specific storage device models, leading to abnormal performance degradation. Finally, we provide insight for system architects to select the appropriate scheme based on the workload and storage device features to fully leverage the advantages of target offloading.

NVMe storage has the advantage of ultra performance. It has gained widespread adoption within cloud data centers, and its significance is only growing. Despite its widespread use, it still faces numerous challenges, such as optimizing performance, managing cost-effectiveness, and ensuring reliability for large-scale deployments. At Alibaba Cloud, we have designed numerous softwarehardware co-design techniques tailored for NVMe storage deployment. These include hardware-assisted NVMe virtualization techniques developed explicitly for bare-metal and virtual machines(VM). BM-Store is a novel hardware-assisted virtualization architecture for bare-metal instances. Furthermore, the CIPU incorporates an embedded virtualization acceleration unit for VM. LightPool, an NVMe-oF-based highperformance storage pool architecture, enhances resource utilization for cloud-native distributed databases. We are also actively pursuing efforts to improve the cost-effectiveness of NVMe SSDs for databases and cloud storage. Additionally, we will delve into the technical challenges and opportunities presented by NVMe storage in the realm of serverless computing and artificial intelligence.

Hardware data prefetching is a well-studied technique to bridge the processor-memory performance gap. Bit-pattern-based prefetchers are one of the most promising spatial data prefetchers that achieve substantial performance gains. In bit-pattern-based prefetchers, the region size is a crucial parameter, which denotes the memory size that can be recorded by a pattern or prefetched by a prediction. However, existing bit-pattern-based prefetchers only support one fixed region size. Our experiment shows that the fixed region size cannot meet the requirements for numerous applications and leads to suboptimal performance and high hardware overhead. In this article, we propose PARS, a pattern-aware spatial data prefetcher supporting multiple region sizes. The key idea of PARS is that it supports multiple region sizes, enabling it to simultaneously enhance application performance while reducing the hardware overhead. Moreover, PARS supports dynamically switching appropriate region sizes for different patterns through an adaptive RS-switching mechanism. We evaluated PARS on numerous workloads and results show that PARS provides an average performance improvement of 40.6% over a baseline with no data prefetchers and outperforms the two state-of-the-art prefetchers Bingo by 2.1% (up to 24.4%) and Pythia by 3.9% (up to 111.2%) in the single-core system. In the four-core system, PARS outperforms Bingo by 5.0% (up to 66.0%) and Pythia by 5.4% (up to 177.9%).