Object tracking is now widely used in sports event broadcasting, security surveillance, and human-computer interaction. It is a challenging task for tracking on unmanned aerial vehicle (UAV) datasets due to many factors such as illumination change, appearance modification, occlusion, motion blur and so on. To solve the problem, a visual correlation filter tracking algorithm based on temporal and spatial regularization is proposed. It employs boolean maps to obtain visual attention, and fuses different features such as color names (CN), histogram of oriented gradient (HOG) and Gray features to enhance the visual representation. New object occlusion judgment method and model update strategy are put forward to make the tracker more robust. The proposed algorithm is compared with other six trackers in terms of distant precision and success rate on UAV123. And the experimental results show that it achieves more stable and robust tracking performance.