In this study, we proposed a fast and accurate PatchMixer-based framework (Patch-TS). After the data were processed, which included missing values and normalization, the environmental factors were selected via the Pearson correlation coefficient method. Then, the newly developed sap flow prediction model was trained. The resulting data demonstrated that the coefficient of determination (R²), mean squared error (MSE), and mean absolute error (MAE) are 0.921, 0.00824, and 0.0497, respectively. The R² of Patch-TS further improved to 0.929 after 7 factors were extracted via the Pearson correlation method. Furthermore, we comparatively analyse the mitigating effects of RevIn (Reversible Instance Normalization) and Dish-TS on data drift. In addition to the predictive performance of the models under different prediction windows, Patch-TS outperforms the other models. The results demonstrate that the model developed in this paper is an effective tool for accurately predicting sap flow, which is a valuable contribution to the practical management of trees and forests.