Low rank and sparse representation (LRSR) with dual-dictionaries-based methods for detecting anomalies in hyperspectral images (HSIs) are proven to be effective. However, the potential anomaly dictionary is vulnerable to being contaminated by the background pixels in the above methods, and this limits the effect of hyperspectral anomaly detection (HAD). In this paper, a dual dictionaries construction method via two-stage complementary decision (DDC–TSCD) for HAD is proposed. In the first stage, an adaptive inner window–based saliency detection was proposed to yield a coarse binary map, acting as the indicator to select pure background pixels. For the second stage, a background estimation network was designed to generate a fine binary map. Finally, the coarse binary map and fine binary map worked together to construct a pure background dictionary and potential anomaly dictionary in the guidance of the superpixels derived from the first stage. The experiments conducted on public datasets (i.e., HYDICE, Pavia, Los Angeles, San Diego-I, San Diego-II and Texas Coast) demonstrate that DDC–TSCD achieves satisfactory AUC values, which are separately 0.9991, 0.9951, 0.9968, 0.9923, 0.9986 and 0.9969, as compared to four typical methods and three state-of-the-art methods.