Agricultural fields are essential in providing human beings with paramount food and other materials. Quick and accurate identification of agricultural fields from the remote sensing images is a crucial task in digital and precision agriculture. Deep learning methods have the advantages of fast and accurate image segmentation, especially for extracting the agricultural fields from remote sensing images. This paper proposed a deep neural network with a dual attention mechanism and a multi-scale feature fusion (Dual Attention and Scale Fusion Network, DASFNet) to extract the cropland from a GaoFen-2 (GF-2) image of 2017 in Alar, south Xinjiang, China. First, we constructed an agricultural field segmentation dataset from the GF-2 image. Next, seven evaluation indices were selected to assess the extraction accuracy, including the location shift, to reveal the spatial relationship and facilitate a better evaluation. Finally, we proposed DASFNet incorporating three ameliorated and novel deep learning modules with the dual attention mechanism and multi-scale feature fusion methods. The comparison of these modules indicated their effects and advantages. Compared with different segmentation convolutional neural networks, DASFNet achieved the best testing accuracy in extracting fields with an F1-score of 0.9017, an intersection over a union of 0.8932, a Kappa coefficient of 0.8869, and a location shift of 1.1752 pixels. Agricultural fields can be extracted automatedly and accurately using DASFNet, which reduces the manual record of the agricultural field information and is conducive to further farmland surveys, protection, and management.