Infrared (IR) small target detection is a crucial component of infrared imaging systems and is vital for applications in surveillance, security, and early warning systems. However, most existing algorithms for detecting small targets in infrared imagery encounter difficulties in achieving both high accuracy and speed, particularly in complex scenes. Additionally, infrared image sequences frequently exhibit gradual background changes as well as sudden alterations, which further complicates the task of detecting small targets. To address these issues, a dual-region search method (DRSM) is proposed and combined with multi-directional filtering, min-sum fusion, and clustering techniques, forming an infrared small moving target detection method in complex scenes. First, a multi-directional filter bank is proposed and it causes the original infrared image sequence to retain only point-like features after the filtering. Then, several consecutive filtered feature maps are superimposed into one, where the moving target will leave a trajectory due to its motion characteristics. Finally, based on the trajectory, a dual-region search strategy is employed to pinpoint the exact location of the target. The experimental outcomes show that, compared to alternative algorithms, the proposed approach outperforms others in terms of detection accuracy and speed, particularly in diverse real-world complex scenarios.