17 December 2024 in ACM Transactions on Reconfigurable Technology and Systems
This article introduces DANSEN , the hardware accelerator component for neoDBMS, a full-stack computational storage system designed to manage on-device execution of database queries/transactions as a Near-Data Processing (NDP)-operation. The proposed system enables Database Management Systems (DBMS) to offload NDP-operations to the storage while maintaining control over data through a native storage interface . DANSEN provides an NDP-engine that enables DBMS to perform both low-level database tasks, such as performing database administration, as well as high-level tasks like executing SQL, on the smart storage device while observing the DBMS concurrency control. Furthermore, DANSEN enables the incorporation of custom accelerators as an NDP-operation (e.g., to perform hardware-accelerated ML inference directly on the stored data). We built the DANSEN storage prototype and interface on an UltraScale+HBM FPGA and fully integrated it with PostgreSQL 12. Experimental results demonstrate that the proposed NDP approach outperforms software-only PostgreSQL using a fast off-the-shelf NVMe drive and significantly improves the end-to-end execution time of an aggregation operation (similar to Q6 from CH-benCHmark, 150 million records) by ≈ 10.6×. The versatility of the proposed approach is also validated by integrating a compute-intensive data analytics application with multi-row results, outperforming PostgreSQL by ≈ 1.5×.
4 April 2025 in IEEE Transactions on Computers
Near-Data Processing (NDP) has been proven useful to accelerate Database Management Systems (DBMS) that handle infrequently accessed data stored in slow persistent storage. A key challenge for such an architecture is the synchronization of host-based and NDP operations, which require fine-grained interactions especially when the NDP device can also update (modify) the DBMS data autonomously. This paper introduces CINDA, the first full-stack computational storage capable of accelerating both read and update (write) database transactions using NDP. The proposed system relies on a hybrid host-device interface to enable the DBMS accessing persisted data, offloading computation to the storage device, and coordinating concurrent device-update operations with the host-update ones. A hybrid interface utilizes a cache-coherent interconnect such as CCIX or CXL for low-latency synchronization using a shared-lock table, and PCIe DMA for high-throughput bulk I/O. We evaluated the effectiveness of the proposed approach in a CCIX-based system by realizing an FPGA-based NDP-capable computational storage device and customizing an NDP-capable DBMS based on PostgreSQL to support update NDP operations. Our full-stack evaluation using the YCSB benchmark demonstrates that CINDA can deliver $\approx$4.2$\times$ end-to-end speedup when executing long-running update transactions directly on the storage device, while the host DBMS performs frequent short updates.
16 March 2021 in Distributed and Parallel Databases
Massive data transfers in modern data-intensive systems resulting from low data-locality and data-to-code system design hurt their performance and scalability. Near-Data processing (NDP) and a shift to code-to-data designs may represent a viable solution as packaging combinations of storage and compute elements on the same device has become feasible. The shift towards NDP system architectures calls for revision of established principles. Abstractions such as data formats and layouts typically spread multiple layers in traditional DBMS, the way they are processed is encapsulated within these layers of abstraction. The NDP-style processing requires an explicit definition of cross-layer data formats and accessors to ensure in-situ executions optimally utilizing the properties of the underlying NDP storage and compute elements. In this paper, we make the case for such data format definitions and investigate the performance benefits under RocksDB and the COSMOS hardware platform.
6 March 2026 in ACM Transactions on Database Systems
The performance and scalability of modern data-intensive systems processing large datasets are limited by unnecessary data movement. Even though near-data processing (NDP) can provably reduce data transfers and increase performance, at present, NDP is utilized primarily in read-only settings. Near-data execution of data-intensive modification operations is currently infeasible due to the lack of transactional consistency and the absence of practicable low-latency synchronization mechanisms between the host database engine and the NDP-engine on smart storage. In this article, we introduce update NDP as an approach to offloading modifications to computational storage with transactional guarantees in an NDP database system called neoDBMS . To ensure consistency, we introduce a low-latency shared lock table between the host and computational storage, based on novel cache-coherent interconnects . We also introduce a novel locking protocol that seamlessly integrates the shared lock table within the lock manager of the host NDP-engine. To handle failure recovery, while preserving high and robust performance, we introduce novel extended locking and logging mechanisms that allow the host and computational storage to perform useful work during log-movement. Our evaluation indicates that in-storage modifications in neoDBMS in mixed workload settings are ≥ 6.52× faster than host-only executions and exhibit robust performance due to lower data movement and better resource utilization.