In this work, the self-excited oscillating pulsating circular pipe is the object of study. Based on the flow evolution characteristics of boundary layer and vortex, the mechanism of enhanced heat transfer by self-excited oscillating pulsating flow is investigated. Moreover, a vital flow structure, the boundary vortex ring (BVR for short), is proposed. The study results show that the vortex evolution within the shear layer inside the self-excited oscillating pulsating chamber has an important influence on the formation of the downstream boundary vortex ring. Both have the same period but different phases. The boundary vortex group formed by the BVR is distributed at intervals in the pipe, and its role in promoting fluid flow increases first and then decreases. At the same time, the strength of the central mainstream area is gradually strengthened. The boundary vortex group's flow state determines the downstream pipe's heat transfer characteristics. The low-velocity zone on both sides determines the position of the heat transfer coefficient enhancement, and the central vorticity determines the amplitude of the enhancement. The boundary vortex group with a complete structure can effectively promote heat transfer, while the boundary vortex group with an incomplete structure can suppress heat transfer. The time-averaged boundary layer thickness increase ratio δ' and the time-averaged equal diameter circular tube performance evaluation index η_T provide the fundamental indexes for designing and optimizing variable cross-section heat transfer circular tubes. Furthermore, the heat transfer coefficient of the tube wall varies synchronously with the thickness of the boundary layer.