Convolution is the most time-consuming operation in modern deep artificial neural networks, so its performance is crucial for fast inference. One of the standard approaches to fast convolution computation is to use GeMM-based convolution algorithms relying on efficient general matrix multiplication (GeMM) from optimized BLAS libraries. However, commonly used GeMM-based algorithms may cause significant memory overhead or avoid it only at the cost of worse performance. In this paper, we propose a novel convolution algorithm, p-im2col, based on a well-known im2col algorithm that avoids memory overhead by splitting a single multiplication of a large matrix into several multiplications of smaller matrices. We theoretically and experimentally compare our algorithm with two other GeMM-based algorithms: im2col, which is widely used as a baseline, and the memory-efficient kn2row-aa. We measure the inference time of these algorithms on central processing units of x86, x86_64, ARM, and MIPS architectures for a large set of convolutional parameters. The proposed algorithm demonstrates a speedup over im2col and kn2row-aa in a number of cases and a significant reduction in additional memory requirements compared to im2col. Based on our experiments, we present a new convolution algorithm selection scheme that considers memory restrictions, CPU architecture, and convolutional parameters and provides a noticeable advantage over each particular algorithm.

Quantized neural networks (QNNs) are widely used to achieve computationally efficient solutions to recognition problems. Overall, eight-bit QNNs have almost the same accuracy as full-precision networks, but working several times faster. However, the networks with lower quantization levels demonstrate inferior accuracy in comparison to their classical analogs. To solve this issue, a number of quantization-aware training (QAT) approaches were proposed. In this paper, we study QAT approaches for two- to eight-bit linear quantization schemes and propose a new combined QAT approach: neuron-by-neuron quantization with straight-through estimator (STE) gradient forwarding. It is suitable for quantizations with two- to eight-bit widths and eliminates significant accuracy drops during training, which results in better accuracy of the final QNN. We experimentally evaluate our approach on CIFAR-10 and ImageNet classification and show that it is comparable to other approaches for four to eight bits and outperforms some of them for two to three bits while being easier to implement. For example, the proposed approach to three-bit quantization of the CIFAR-10 dataset results in 73.2% accuracy, while baseline direct and layer-by-layer result in 71.4% and 67.2% accuracy, respectively. The results for two-bit quantization for ResNet18 on the ImageNet dataset are 63.69% for our approach and 61.55% for the direct baseline.

Quantization is a widespread method for reducing the inference time of neural networks on mobile Central Processing Units (CPUs). Eight-bit quantized networks demonstrate similarly high quality as full precision models and perfectly fit the hardware architecture with one-byte coefficients and thirty-two-bit dot product accumulators. Lower precision quantizations usually suffer from noticeable quality loss and require specific computational algorithms to outperform eight-bit quantization. In this paper, we propose a novel 4.6-bit quantization scheme that allows for more efficient use of CPU resources. This scheme has more quantization bins than four-bit quantization and is more accurate while preserving the computational efficiency of the later (it runs only 4% slower). Our multiplication uses a combination of 16- and 32-bit accumulators and avoids multiplication depth limitation, which the previous 4-bit multiplication algorithm had. The experiments with different convolutional neural networks on CIFAR-10 and ImageNet datasets show that 4.6-bit quantized networks are 1.5–1.6 times faster than eight-bit networks on the ARMv8 CPU. Regarding the quality, the results of the 4.6-bit quantized network are close to the mean of four-bit and eight-bit networks of the same architecture. Therefore, 4.6-bit quantization may serve as an intermediate solution between fast and inaccurate low-bit network quantizations and accurate but relatively slow eight-bit ones.

RFD-like binary descriptors have been designed to be fast and demonstrate good quality in image-matching tasks. One of those descriptors, RFDoc, produces state-of-the art results when applied to document localization systems. However, the computational efficiency of such descriptors strongly depends on their implementation. In this study, we consider the computation of RFD-like descriptors for an 8-bit single-channel image; provide a detailed implementation of the baseline algorithm; demonstrate its weak points; and propose four modifications. In those modifications, we investigate two ways to accelerate the descriptor computations: 1) compute common operations globally for the entire input image instead of computing them locally for every patch; or 2) use lookup tables to replace the most computationally demanding operations and minimize the number of conversions between integer and floating point types. Experiments on the document identification and localization task on the MIDV-2020 dataset have shown that the modifications with lookup tables are noticeably faster than the baseline, achieving a 2-2.6 times acceleration on x86 and ARM CPUs. Based on experimental results, modifications with global operations may outperform the baseline algorithm if there are many intersecting patches that require descriptor computation. We also demonstrated that one can use any of the proposed algorithms without loss of image-matching quality and without necessity to retrain the parameters of RFD-like descriptors. Finally, we propose an efficient way to compute the descriptors for four orientations of a patch, which is a important for document location systems. The proposed method reduces a common computation part for four orientations to a single run; thus, four descriptors are computed 3 times faster than the direct computation of the descriptors for four patches, as shown experimentally.

This paper focuses on the computational optimization of RANSAC. We describe the Parallel Efficient Sample Consensus (PESAC) framework that allows efficient utilization of SIMD extensions and provides memory locality due to a special way of storing the input sequence of correspondences and generating a batch of samples per one main loop iteration. It is inspired by the USAC framework and has a block structure capable of implementing most modern RANSAC-based methods. We enhance it with individual blocks of sample and model restrictors that are aimed at the rejection of “bad” samples and model hypothesis before time-consuming model computation and verification blocks. We also provide a detailed description implementing 2D homography estimation problem in PESAC and benchmark the running time on the MIDV-2020 dataset of identity documents. Comparing to naive implementation, we accelerated our framework by 122 times for the document classification task (with a 6 % increase in accuracy) and by 18 times for document tracking (with a 46 % decrease in tracking failure rate) by using both restrictors and vector processing. This version also outperformed a number of USAC implementations from OpenCV-4.6.0 in runtime and accuracy of estimation (3 times faster, 6 % greater accuracy for the classification task, and 2 times faster, 33 % lower failure rate for tracking if comparing with USAC_MAGSAC).

This work is devoted to methods for creating fast and accurate neural network algorithms for central processors, which were proposed by scientists of the V.L. Arlazarov’s scientific school. It outlines general principles and approaches to improving computational efficiency and discusses specific examples: tensor convolution decompositions that simplify convolutional neural networks; bounded nonlinear activation function ratio, which is calculated faster than exponential activation functions; and p-im2col convolution algorithm, which allows you to achieve a balance between computational efficiency and RAM consumption. Particular attention is paid to quantized (8- and 4-bit integer) neural networks, their training, implementation, and limitations on some central processor architectures, such as Elbrus.