28 April 2025 in Pattern Recognition
5 November 2025 in TechRxiv
Efficient deep learning depends on high-performance tensor programs, especially in on-device mobile computing. In recent years, automated tensor program optimization, known as tensor compilation, has become the primary approach for generating efficient tensor programs. However, a core challenge in real-time optimizing on-device systems remains: how to generate higher-performance tensor programs within shorter time. This paper presents E 2 Gensor, an enhanced elastic graph-based constructive tensor compilation method, which builds upon the core principles of Gensor while introducing comprehensive extensions and enhancements tailored for mobile computing environments. Unlike existing tree-based methods, E 2 Gensor models the construction space as a graph structure and employs probabilistic modeling via Markov analysis for selecting scheduling primitives, thereby discovering higher-performance solutions while maintaining the rapid optimization efficiency. To address the constraints of limited memory, E 2 Gensor introduces a probability adapter and a noval scheduling primitive called double buffer to improve memory utilization and data transfer efficiency. Furthermore, we develop a just-in-time end-to-end optimization system for real-time on-device scenarios with dynamic input shapes based on E 2 Gensor. Evaluations show that E 2 Gensor consistently outperforms existing methods across various mobile GPU platforms, achieving an average performance improvement of 12% and a peak improvement of up to 20% compared to Gensor, while exhibiting excellent practicality and adaptability in mobile computing scenarios.