Existing databases supporting Online Transaction Processing (OLTP) workloads based on non-volatile memory (NVM) have not fully leveraged hardware characteristics, resulting in an imbalance between throughput and recovery performance. In this paper, we conclude with the reason why existing designs fail to achieve both: placing indexes on NVM results in numerous random writes and write amplification for index updates, leading to a decrease in system performance. Placing indexes on dynamic random access memory (DRAM) results in much time consumption for rebuilding indexes during recovery. To address this issue, we propose FIR, an NVM OLTP Engine with the fast rebuilding of the DRAM indexes, achieving instant system recovery while maintaining high throughput. Firstly, we design an index checkpoint strategy. During recovery, the indexes are quickly rebuilt by the bottom-up algorithm with index checkpoints. Then, to achieve instant recovery of the entire engine after rebuilding indexes, we optimize the existing log-free design by leveraging time-ordered storage, which significantly reduces the number of NVM writes. We also implement garbage collection based on data redistribution, enhancing system availability. The experimental results demonstrate that FIR achieves 98% of the performance of state-of-the-art OLTP Engine when running TPCC and YCSB. And the recovery speed of FIR is 43.6×–54.5× faster, achieving near-instantaneous recovery.

Most modern in-memory database systems rely on multi-version concurrency control (MVCC) to support real-time data analysis without interfering with concurrent writes. While MVCC elegantly enables lightweight isolation of readers and writers, it also generates many short-lived versions and results in long version chains. Existing version storage designs employ traditional data layouts lacking version information. Consequently, as time elapses, the long-running query latency increases due to increased chain lengths. In this paper, we introduce a novel version-aware data layout for in-memory MVCC databases. This layout improves search performance for heterogeneous workloads. It differs from previous approaches by organizing data in a version-centric manner, which supports index-only searches and reduces costly version traversals. Specifically, the layout utilizes a version B⁺-tree to organize the latest tuple versions of a table, making the table index-organized. Additionally, it employs range partitioning and epoch partitioning techniques to group older tuple versions efficiently. This design allows for fast retrieval of the correct versions visible to transactions, minimizing redundant memory accesses. Compared to existing version storage schemes, the version-aware data layout considers workload access patterns and the opportunity to skip unnecessary version accesses. The experimental evaluation indicates that the version-aware data layout can outperform the state-of-the-art designs by 1.15× ∼ 2×.

Large-scale data-intensive applications need massive real-time data processing. Recent hybrid DRAM-PM main memory database systems provide an effective approach by persisting data to persistent memory (PM) in an append-based manner for efficient storage while maintaining the primary database copy in DRAM for high throughput rates. However, they fail to achieve high performance under a hybrid workload because they are unaware of the impact of pointer chasing. In this work, we investigate the impact of chasing pointers on modern main memory database systems to eliminate this bottleneck. We propose Index-Organized data layout that supports efficient reads and updates. We combine two techniques, i.e., cacheline-aligned node layout and cache prefetching, to accelerate pointer chasing, reducing memory access latency. We present four optimizations, i.e., pending versions, fine-grained memory management, Index-SSN, and cacheline-aligned writes, for supporting efficient transaction processing and fast logging. We implement our proposed data layout based on an open-sourced main memory database system. We conduct extensive evaluations on a 20-core machine equipped with Intel Optane DC Persistent Memory Modules. Experimental results demonstrate that Index-Organized obtains up to 3x speedup than the conventional data layouts, i.e., row-store, column-store, and row+column.

Most modern in-memory database systems rely on multi-version concurrency control to support real-time data analysis without interfering with concurrent writes. However, this is not a good fit for heterogeneous workloads. We find that long version chains are the root cause of the throughput reduction. In this paper, we exploit a scalable version-aware data partitioning and placement approach for heterogeneous workloads that incorporates a suite of optimized techniques to significantly reduce the overheads incurred both during the initial placement and during version ingestion at runtime. The experiment results show that the proposed approach achieves 2x performance improvement compared with existing state-of-the-art approaches.

Existing databases supporting Online Transaction Processing (OLTP) workloads based on non-volatile memory (NVM) almost all use Multi-Version Concurrency Control (MVCC) protocol to ensure data consistency. MVCC allows multiple transactions to execute concurrently without lock conflicts, reducing the wait time between read and write operations, and thereby significantly increasing the throughput of NVM OLTP engines. However, it requires garbage collection (GC) to clean up the obsolete tuple versions to prevent storage overflow, which consumes additional system resources. Furthermore, existing GC approaches in NVM OLTP engines are inefficient because they are based on methods designed for dynamic random access memory (DRAM) OLTP engines, without considering the significant differences in read/write bandwidth and cache line size between NVM and DRAM. These approaches either involve excessive random NVM access (traversing tuple versions) or lead to too many additional NVM write operations, both of which degrade the performance and durability of NVM. In this paper, we propose TB-Collect, a high-performance GC approach specifically designed for NVM OLTP engines. On the one hand, TB-Collect separates tuple headers and contents, storing data in an append-only manner, which greatly reduces NVM writes. On the other hand, TB-Collect performs GC at the block level, eliminating the need to traverse tuple versions and improving the utilization of reclaimed space. We have implemented TB-Collect on DBx1000 and MySQL. Experimental results show that TB-Collect achieves 1.15 to 1.58 times the throughput of existing methods when running TPCC and YCSB workloads.