As an essential part of current mainstream computing systems, GPUs are not only powerful graphics engines but also highly parallel programmable processors. Collaboration between CPUs and GPUs is required to obtain high computing performance in multi-CPU and multi-GPU heterogeneous systems. It is challenging to develop new parallel algorithms on heterogeneous architectures with multiple CPUs and multiple GPUs for such purposes as communication, load balancing, memory spaces, and synchronization We present a parallel Cholesky block factorization algorithm for heterogeneous multi-CPU and multi-GPU architectures. First, a matrix is partitioned into different-sized blocks based on with the performance of the CPU and GPU. Then, a one-dimensional row block-cyclic distribution strategy is used to allocate row block data to every CPU and GPU to minimize communication. The computing task related to the definite row block will then be executed by the corresponding CPU or GPU. Experiments on a system with two CPUs and eight GPUs show good load balancing, parallelism, communication cost, and scalability.

This paper explores a device-to-device (D2D) assisted setting in the multi-access networks, where the users communicate with each other and each one can access multiple neighbouring cache nodes in a cyclic wrap-around fashion. For such networks, we introduce two new combinatorial arrays namely the caching array and the delivery array to characterize the coded caching process. Based on this, a novel construction of caching array and delivery array is derived through a smart transform over the existing placement delivery array (PDA). The resulting coded caching scheme can achieve a small transmission rate under the case of uncoded placement. It is shown that the proposed scheme has advantages in both the transmission rate and the subpacketization level over the benchmark scheme.

The efficient development of system software and design applications in parallel architecture is a notable challenge considering various aspects, such as load balancing, memory spaces, communication, and synchronization. This paper presents a block parallel Cholesky factorization algorithm for a multicore system, which is developed based on activity on edge network. First, the basic block computing tasks and their dependencies are taken as vertices and edges, respectively, and a directed acyclic graph corresponding to the specific block parallel Cholesky factorization is generated. Next, each edge of the directed acyclic graph is assigned to a weight equal to the processing time of the initial vertex of the edge, and the directed acyclic graph becomes an activity on edge network with only one starting and one ending vertex. Finally, a queuing algorithm is designed for the basic block computing tasks according to the edge activity on edge network, and a dynamic scheduling strategy is developed for block parallel Cholesky factorization. The results of the experiments concerning the parallel execution time of the algorithm in multicore systems with different configurations demonstrate that the proposed algorithm has notable advantages compared with the traditional static scheduling algorithm, and it exhibits satisfactory load balancing, parallelism, and scalability capacities.

Parallel computing is increasingly important in computer architectures, parallel architecture has become ubiquitous in our everyday lives. Novel architectures and programming models pose new challenges to algorithm design and system software development. This paper presents a two-stage column block parallel LU factorization algorithm for multiple-processor architectures. Any given matrix is first partitioned into large blocks, and then, every large block is partitioned into a number of small blocks according to the number of processors. Finally, the small column blocks are allocated to processors in an orderly “serpentine arrangement.” Each iteration of the column block parallel LU factorization is separated into two stages of operation. In the first stage, the first-step factorization operation is processed in advance and nonblocking communication is used to reduce the processor idle and waiting time and improve parallelism. In the second stage, the large blocks are used to satisfy more powerful processors, such as GPUs, which require more data to exploit their computing capabilities. Experiments are conducted on a multicore system and multi-GPU system with different configurations to test the algorithm’s performance. Compared with other related column block parallel LU factorizations, the two-stage algorithm exhibits better load balancing and parallel execution time performance.

Most supercomputers are shipped with both a CPU and a GPU. With the powerful parallel computing capability of GPUs, heterogeneous computing architecture produces new challenges for system software development and application design. Because of the significantly different architectures and programming models of CPUs and GPUs, conventional optimization techniques for CPUs may not work well in a heterogeneous multi‐CPU and multi‐GPU system. We present a heterogeneous parallel LU factorization algorithm for heterogeneous architectures. According to the different performances of the processors in the system, any given matrix is partitioned into different sizes of basic column blocks. Then, a static task allocation strategy is used to distribute the basic column blocks to corresponding processors uniformly. The idle time is minimized by optimized sizes and the number of basic column blocks. Right‐looking ahead technology is also used in systems configured with one CPU core to one GPU to decrease the wait time. Experiments are conducted to test the performance of synchronization and load balancing, communication cost, and scalability of the heterogeneous parallel LU factorization in different systems and compare it with the related matrix algebra algorithm on a heterogeneous system configured with multiple GPUs and CPUs.