Although massive pruning methods are emerging for achieving structural sparsity in convolutional neural networks (CNN), most of them target structures such as ResNet. Meanwhile, previous works take more interest in pruning filters inside a residual block and keep the shortcut connection intact, leading to an imbalanced network structure. In this paper, we focus on the penalty-based method to prune already compact networks. In contrast to the broadly used $L_{1}$ constraint, which shrinks the parameters uniformly, we propose a novel penalty term that is similar in shape to an upside-down Laplace distribution. The penalty allows us to impose more pressure on potential weak channels but protects others during training to avoid damaging crucial channels, especially for compact architectures. We also design a candidate selection strategy to cooperate with the penalty-based training procedure. Besides, we address the residual block pruning problem by a scaling factor elimination skill, which is often ignored in other research. Our method reduces 50% parameters of MobileNet v1/v2 with a tolerable accuracy degradation. We further conduct pruning on MobileNetv1-SSDLite to compress parameters by 60%, manifesting the ability to generalize to different visual tasks. The experiment results demonstrate that our method outperforms pruning frameworks based on channel importance without a complicated tuning for hyper-parameters like search-based methods.
Image demosaicking has been an essential and challenging problem among the most crucial steps of image processing behind image sensors. Due to the rapid development of intelligent processors based on deep learning, several demosaicking methods based on a convolutional neural network (CNN) have been proposed. However, it is difficult for their networks to run in real-time on edge computing devices with a large number of model parameters. This paper presents a compact demosaicking neural network based on the UNet++ structure. The network inserts densely connected layer blocks and adopts Gaussian smoothing layers instead of down-sampling operations before the backbone network. The densely connected blocks can extract mosaic image features efficiently by utilizing the correlation between feature maps. Furthermore, the block adopts depthwise separable convolutions to reduce the model parameters; the Gaussian smoothing layer can expand the receptive fields without down-sampling image size and discarding image information. The size constraints on the input and output images can also be relaxed, and the quality of demosaicked images is improved. Experiment results show that the proposed network can improve the running speed by 42% compared with the fastest CNN-based method and achieve comparable reconstruction quality as it on four mainstream datasets. Besides, when we carry out the inference processing on the demosaicked images on typical deep CNN networks, Mobilenet v1 and SSD, the accuracy can also achieve 85.83% (top 5) and 75.44% (mAP), which performs comparably to the existing methods. The proposed network has the highest computing efficiency and lowest parameter number through all methods, demonstrating that it is well suitable for applications on modern edge computing devices.
Nowadays, the vision chip bridging sensing and processing has been extensively employed in high-speed image processing, owing to its excellent performance, low power consumption, and economical cost. However, there is a dilemma in designing processors to support conventional computer vision algorithms and neural networks since the two algorithms have a non-trivial trade-off in proposing a unified architecture. By analyzing computation properties, we propose a novel hierarchical parallel vision processor (ViP) for hybrid vision chips to accelerate both traditional computer vision (CV) and neural network (NN). The ViP architecture includes three parallelism levels: PE for pixel-centric, computing core (CC) for block, and vision core (VC) for global. PEs contain dedicated computing units and data paths for convolution operations without degrading its flexibility. Each CC is driven by customized SIMD instructions and can be dynamically connected for meeting block parallelism requirements. ViP is fabricated in 65nm CMOS technology and achieves a peak performance of 614.4 GOPS and an energy efficiency of 640 GOPS/W at 200 MHz clock frequency. Notably, several experiments on CV and NN are performed, illustrating an ultra-low latency in executing hybrid algorithms.
4 November 2021 in IEEE Transactions on Circuits and Systems II: Express Briefs
With the development of artificial intelligence research, the convolutional neural networks (CNN) triumphantly continue to conquer diverse visual tasks, with lots of technical and practical problems emerging. When the detection networks need to meet the high-speed processing requirement and the low power consumption constraint in edge computing, dedicated circuits and algorithm optimization are necessary. In this brief, we propose a non-maximum suppression (NMS) coprocessor that cooperated with a Vision Processor (VP) to speed up the detection. It contains a parallel process unit and dynamic memory management to accelerate NMS processing. Meanwhile, the software workflow provides a comprehensive method to convert the one-stage detection network to our processor in a post-training manner. The experimental results show that our co-design solution reaches 166 FPS to run a MobileNet-SSD, promising to deploy in edge-computing scenarios. Moreover, the detection accuracy on the Airbus dataset is 88.6%, favorable in practical applications.
8 June 2022 in IEEE Transactions on Circuits and Systems II: Express Briefs
Vision chips perform image capture and real-time intelligent image processing by integrating an imager and a vision processor on a single chip, having broad application prospects. This brief proposes a programmable and flexible vision processor with a dual-issue micro-architecture. The processor consists of a reconfigurable vector unit, a flexible memory access network, and a non-maximum suppression (NMS) block. It can efficiently implement both deep neural network (DNN) and traditional computer vision (CV) algorithms. The vector unit performs single-instruction multiple-vector (SIMV) parallel operations with reconfigurable vector width. The flexible memory access network adaptively supports multiple vector operations under different vector widths. A four-MAC processing element (PE) in the vector unit is designed to increase computational power and data reuse rate. The NMS block can speed up the object location processing of the detection networks. The chip is fabricated in a 28nm process. The experimental results show that the maximum clock frequency, peak performance, and peak energy efficiency are 600MHz, 1.2TOPS, and 2.03TOPS/W, respectively. The Mobilenet-V1 processing achieves a throughput of 404 fps under a $256\times 224$ image size and an 87.15%(top-5) accuracy on the ImageNet dataset.