Breast cancer is the leading cause of cancer death in women, and early detection can reduce mortality. Architectural distortion (AD) is a feature of clinical manifestations for breast cancer, however, due to its complex structure and low detection accuracy, which cause a high mortality of breast cancer. In order to improve the accuracy of AD detection and reduce the mortality of breast cancer, this paper proposes a new method by combining the non-subsampled contourlet transform (NSCT) with the improved pulse coupled neural network (PCNN). Firstly, the top–bottom hat transformation and the exponential transformation are employed to enhance the image. Secondly, the NSCT is employed to expand the overall contrast of the mammograms and filter out the noise. Finally, the improved PCNN by the maximum inter-class variance threshold selection method is employed to complete the AD detection. This proposed approach is tested on the public and authoritative database—Digital Database for Screening Mammography (DDSM). The specificity of the method is 98.73%, the accuracy is 93.16%, and the F1-score is 79.80%, and the area under curve (AUC) of the receiver operating characteristic (ROC) curve is 0.93, these results clearly demonstrate that the proposed method is comparable with those methods in recent literatures. This proposed method is simple, furthermore it can achieve high accuracy and help doctors to perform computer-aided detection of AD effectively.