Change detection, as an important task of remote sensing image processing, has a wide range of applications in many aspects such as land use and natural disaster assessment. Recent change detection methods have achieved good results. However, due to the environmental difference between the bi-temporal images and the complicated imaging condition, there are usually problems such as missing small objects, incomplete objects, and rough edges in the change detection results. The existing change detection methods usually lack attention in these areas. In this paper, we propose a Siamese change detection method, named SMD-Net, for bi-temporal remote sensing change detection. The proposed model uses multi-scale difference maps to enhances the information of the changed areas step by step in order to have better change detection results. Furthermore, we propose a Siamese residual multi-kernel pooling module (SRMP) for high-level features to enhance the high-level change information of the model. For the low-level features of multiple skip connections, we propose a feature difference module (FDM) that uses feature difference to fully extract the change information and help the model generate more accurate details. The experimental results of our method on three public datasets show that compared with other benchmark methods, our network comprises better effectiveness and has a better trade-off between accuracy and calculation cost.