Artificial intelligence technologies based on deep convolutional neural networks and large language models have made significant breakthroughs in many tasks, such as image recognition, target detection, semantic segmentation, and natural language processing, but also face a conflict between the high computational capacity of the algorithms and limited deployment resources. Quantization, which converts floating-point neural networks into low-bit-width integer networks, is an important and essential technique for efficient deployment and cost reduction in edge computing. This paper analyzes various existing quantization methods, showcases the deployment accuracy of advanced techniques, and discusses the future challenges and trends in this domain.

Artificial intelligence technologies based on deep convolutional neural networks and large lan-guage models have made significant breakthroughs in many tasks such as image recognition, target detection, semantic segmentation and natural language processing, but also face the problem of contradiction between the high computational capacity of the algorithms and the limited deployment resources. Quantization, which converts floating-point neural networks into low-bit-width integer networks, is an important and essential technique for efficient deploy-ment and cost reduction in edge computing. This paper analyze various existing quantization methods, showcase the deployment accuracy of advanced techniques, and discuss the future challenges and trends in this domain.

Quantization plays a crucial role in deploying neural network models on resource-limited hardware. However, current quantization methods have issues like large accuracy loss and poor generalization for complex tasks. These issues pose obstacles to the practical application of deep learning and large language models in smart systems. The main problem is our limited understanding of quantization's effect on accuracy, and there is also a need for more effective approaches to evaluate the performance of the quantized models. To address these concerns, we develop a novel method that leverages the self-attention mechanism. This method predicts a quantized model's accuracy using a single representative image from the test set. It utilizes the Transformer encoder and decoder to perform this prediction. The prediction error of the quantization accuracy on three types of neural network models is only 2.35%. The proposed method enables rapid performance assessment of the quantized models during the development stage, thereby facilitating the optimization of quantization parameters and promoting the practical application of neural network models.

Quantization plays a crucial role in deploying neural network models on resource-limited hardware. However, current quantization methods have issues like the large accuracy loss and poor generalization for complex tasks. These issues pose obstacles to the practical application of deep learning and large language models in smart systems. The main problem is our limited understanding of quantization’s effect on accuracy, and there is also a need for more effective approaches to evaluate the performance of the quantized models. To address these concerns, we develop a novel method that leverages the self-attention mechanism. This method predicts a quantized model’s accuracy using a single representative image from the test set. It utilizes the transformer encoder and decoder to perform this prediction. The prediction error of the quantization accuracy on three types of neural network models is 2.44%. The proposed method enables rapid performance assessment of the quantized models during the development stage, thereby facilitating the optimization of the quantization parameters and promoting the practical application of neural network models.

Quantization reduces the precision of neural network parameters to accelerate inference and lower power consumption, but it often causes noticeable accuracy degradation. We propose a differentiable quantization framework that replaces the non-differentiable rounding operation with a continuous surrogate function. During QAT, gradients are backpropagated through the proposed surrogate rather than being estimated by the STE, enabling gradient-based optimization of model weights, quantization parameters, and layer-wise bit-width configurations. Experiments on CIFAR-10 show that our method achieves higher accuracy than several representative quantization approximation methods under different bit-width settings. On embedded platforms, it improves post-quantization accuracy by up to 3.66 percentage points over industrial quantization frameworks such as TensorRT and Huawei AMCT on detection and segmentation tasks, and outperforms representative bit-width allocation methods by up to 7.49 percentage points. These results demonstrate the effectiveness of the proposed method for improving the accuracy of quantized neural networks on resource-constrained devices.