9 November 2023 in ACM Transactions on Embedded Computing Systems
Transposed convolution has been prevailing in convolutional neural networks (CNNs), playing an important role in multiple scenarios such as image segmentation and back-propagation process of training CNNs. This mainly benefits from the ability to up-sample the input feature maps by interpolating new information from the input feature pixels. However, the backward-stencil computation constrains its performance and hindered its wide application in diverse platforms. Moreover, in contrast to the efforts on accelerating the convolution, there is a rare investigation on the acceleration of transposed convolution that is identically compute-intensive as the former. For acceleration of transposed convolution, we propose an intermediate-centric dataflow scheme, in which we decouple the generation of the intermediate patch from its further process, aim at efficiently performing the backward-stencil computation . The intermediate-centric dataflow breaks the transposed convolution into several phases/stages, achieving feeding the input feature maps and performing the backward-stencil computation in a pipelining manner. It also provides four-degree computation parallelism and efficient data reuse of input feature maps/weights. Furthermore, we also theoretically analyze the irregular data dependence leveraging the polyhedral model, which constrains the parallel computing of transposed convolution. Additionally, we devise an optimization problem to explore the design space and automatically generate the optimal design configurations for different transposed convolutional layers and hardware platforms. By selecting the representative transposed convolutional layers from DCGAN, FSRCNN, and FCN, we generate the corresponding accelerator arrays of intermediate-centric dataflow on the Xilinx Alveo U200 platform and reach the performance of 3.92 TOPS, 2.72 TOPS, and 4.76 TOPS, respectively.
19 October 2023 in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems
In modern times, domain-specific accelerators require numerous functional components to execute complex applications in a particular domain. To ensure efficient development, the conventional approach involves decomposing, implementing, and integrating modules. Over the past decade, the generator-based method has proven to enhance the productivity of module implementation. However, current abstractions pose challenges for integrating modules implemented by generators, due to implicit interface definitions, nonunified performance modeling, and fragmented memory management. These limitations result in a lower productivity of the integration process and decreased performance of the integrated accelerators. To overcome these drawbacks, we propose Weave, an abstraction for integrating generated modules and an agile design flow for domain-specific accelerators. The Weave abstraction guides module implementation and integration with a unified performance model and memory management. Furthermore, the Weave integration flow, consisting of generation, selection, and integration phrases, enables optimization of the performance of the integrated accelerator with a design space exploration algorithm and hierarchical memory management. In the experiments, the accelerator developed by Weave achieves $1.93\times $ higher performance in the deep learning domain compared to an open-source accelerator, and the integrated accelerator maintains performance for various applications with different memory access patterns.