14 December 2020 in IEEE Transactions on Parallel and Distributed Systems
Modern multiprocessor systems running multiple applications concurrently exhibit irregular memory access pattern during different phases of execution. The principle of locality is hard to exploit in the presence of such irregular memory requests and may result in additional delays due to resource conflicts throughout memory hierarchy. Prefetching is a promising technique to reduce the memory access latency where data is speculatively fetched ahead of time and stored in a faster memory structure like cache or dedicated prefetch buffer. The emergence of 3D-stacked DRAM provides huge internal bandwidth that makes memory-side prefetching an effective approach to improving system performance. Leveraging the unique architecture of 3D-stacked DRAM, we introduce a memory-side prefetching scheme that works in conjunction with dynamic page mode to reduce memory access latency. We introduce a novel prefetch buffer management scheme that makes intelligent replacement decision based on the utilization and recency of the prefetched data, which also serves as a guidance for future prefetching. Simulation results indicate that our approach improves performance by 21.8 percent on average, compared to a baseline scheme that prefetches a whole row on consecutive hits and implements static open page policy. Our scheme also outperforms an existing memory-side prefetching scheme by 13.2 percent on average, which dynamically adjusts the prefetch degree based on the usefulness of prefetched data.
10 May 2023 in ACM Transactions on Design Automation of Electronic Systems
Non-volatile memory has been extensively researched as the alternative for a DRAM-based system; however, the traditional memory controller cannot efficiently track and schedule operations for all the memory devices in heterogeneous systems due to different timing requirements and complex architecture supports of various memory technologies. To address this issue, we propose a hybrid memory architecture framework called POMI (POlling-based Memory Interface). It uses a small buffer chip inserted on each DIMM (Dual In-line Memory Module) to decouple operation scheduling from the controller to enable the support for diverse memory technologies in the system. Unlike the conventional DRAM-based system, POMI uses a polling-based memory bus protocol for communication and to resolve any bus conflicts between memory modules. The buffer chip on each DIMM will provide feedback information to the main memory controller so that the polling overhead is trivial. We propose two unique designs. The first one adds additional bus lines for sending the feedback information, and the second one utilizes the Command/Address bus. The framework provides several benefits: a technology-independent memory system, higher parallelism, and better scalability. Our experimental results show that POMI can efficiently support both homogeneous and heterogeneous systems. Compared with the conventional DDR4-2400 implementation, our scheme improves the performance of memory-intensive workloads by 3.7% on average. Compared with an existing interface for hybrid memory systems, Twin-Load, it also improves performance by 22.0% on average for memory-intensive workloads.
20 November 2023 in IEEE Computer Architecture Letters
DRAM memory reliability is increasingly a concern as recent studies found. In this paper, we propose RAIMD (Redundant Array of Independent Memory Devices), an energy-efficient memory organization with RAID-like error protection. In this organization, each memory device works as an independent memory module to serve a whole memory request and to support error detection and error recovery. It relies on the high data rate of modern memory device to minimize the performance impact of increased data transfer time. RAIMD provides chip-level error protection similar to ChipKill but with significant energy savings. Our simulation results indicate that RAIMD can save memory energy by 26.3% on average with a small performance overhead of 5.3% on DDR5-4800 memory systems for SPEC2017 multi-core workloads.