This study describes a novel three-dimensional (3D) convolutional neural networks (CNN) based method that automatically classifies crops from spatio-temporal remote sensing images. First, 3D kernel is designed according to the structure of multi-spectral multi-temporal remote sensing data. Secondly, the 3D CNN framework with fine-tuned parameters is designed for training 3D crop samples and learning spatio-temporal discriminative representations, with the full crop growth cycles being preserved. In addition, we introduce an active learning strategy to the CNN model to improve labelling accuracy up to a required threshold with the most efficiency. Finally, experiments are carried out to test the advantage of the 3D CNN, in comparison to the two-dimensional (2D) CNN and other conventional methods. Our experiments show that the 3D CNN is especially suitable in characterizing the dynamics of crop growth and outperformed the other mainstream methods.

Accurate crop distribution maps provide important information for crop censuses, yield monitoring and agricultural insurance assessments. Most existing studies apply low spatial resolution satellite images for crop distribution mapping, even in areas with a fragmented landscape. Unmanned aerial vehicle (UAV) imagery provides an alternative imagery source for crop mapping, yet its spectral resolution is usually lower than satellite images. In order to produce more accurate maps without losing any spatial heterogeneity (e.g., the physical boundary of land parcel), this study fuses Sentinel-2A and UAV images to map crop distribution at a finer spatial scale (i.e., land parcel scale) in an experimental site with various cropping patterns in Heilongjiang Province, Northeast China. Using a random forest algorithm, the original, as well as the fused images, are classified into 10 categories: rice, corn, soybean, buckwheat, other vegetations, greenhouses, bare land, water, roads and houses. In addition, we test the effect of UAV image choice by fusing Sentinel-2A with different UAV images at multiples spatial resolutions: 0.03 m, 0.10 m, 0.50 m, 1.00 m and 3.00 m. Overall, the fused images achieved higher classification accuracies, ranging between 10.58% and 16.39%, than the original images. However, the fused image based on the finest UAV image (i.e., 0.03 m) does not result in the highest accuracy. Instead, the 0.10 m spatial resolution UAV image produced the most accurate map. When the spatial resolution is less than 0.10 m, accuracy decreases gradually as spatial resolution decreases. The results of this paper not only indicate the possibility of combining satellite images and UAV images for land parcel level crop mapping for fragmented landscapes, but it also implies a potential scheme to exploit optimal choice of spatial resolution in fusing UAV images and Sentinel-2A, with little to no adverse side-effects.

Precise crop classification from multi-temporal remote sensing images has important applications such as yield estimation and food transportation planning. However, the mainstream convolutional neural networks based on 2D convolution collapse the time series information. In this study, a 3D fully convolutional neural network (FCN) embedded with a global pooling module and channel attention modules is proposed to extract discriminative spatiotemporal presentations of different types of crops from multi-temporal high-resolution satellite images. Firstly, a novel 3D FCN structure is introduced to replace 2D FCNs as well as to improve current 3D convolutional neural networks (CNNs) by providing a mean to learn distinctive spatiotemporal representations of each crop type from the reshaped multi-temporal images. Secondly, to strengthen the learning significance of the spatiotemporal representations, our approach includes 3D channel attention modules, which regulate the between-channel consistency of the features from the encoder and the decoder, and a 3D global pooling module, which selects the most distinctive features at the top of the encoder. Experiments were conducted using two data sets with different types of crops and time spans. Our results show that our method outperformed in both accuracy and efficiency, several mainstream 2D FCNs as well as a recent 3D CNN designed for crop classification. The experimental data and source code are made openly available at http://study.rsgis.whu.edu.cn/pages/download/.

Unmanned aerial vehicle (UAV) and structure from motion (SfM) photogrammetry techniques are widely used for field-based, high-throughput plant phenotyping nowadays, but some of the intermediate processes throughout the workflow remain manual. For example, geographic information system (GIS) software is used to manually assess the 2D/3D field reconstruction quality and cropping region of interests (ROIs) from the whole field. In addition, extracting phenotypic traits from raw UAV images is more competitive than directly from the digital orthomosaic (DOM). Currently, no easy-to-use tools are available to implement previous tasks for commonly used commercial SfM software, such as Pix4D and Agisoft Metashape. Hence, an open source software package called easy intermediate data processor (EasyIDP; MIT license) was developed to decrease the workload in intermediate data processing mentioned above. The functions of the proposed package include (1) an ROI cropping module, assisting in reconstruction quality assessment and cropping ROIs from the whole field, and (2) an ROI reversing module, projecting ROIs to relative raw images. The result showed that both cropping and reversing modules work as expected. Moreover, the effects of ROI height selection and reversed ROI position on raw images to reverse calculation were discussed. This tool shows great potential for decreasing workload in data annotation for machine learning applications.