Global agriculture is facing the challenge of a phenotyping bottleneck due to large-scale screening/breeding experiments with improved breeds. Phenotypic analysis with high-throughput, high-accuracy and low-cost technologies has therefore become urgent. Recent advances in image-based 3D reconstruction offer the opportunity of high-throughput phenotyping. The main aim of this study was to quantify and evaluate the canopy structure of plant populations in two and three dimensions based on the multi-view stereo (MVS) approach, and to monitor plant growth and development from seedling stage to fruiting stage. Multi-view images of flat-leaf cucumber, small-leaf pepper and curly-leaf eggplant were obtained by moving a camera around the plant canopy. Three-dimensional point clouds were reconstructed from images based on the MVS approach and were then converted into surfaces with triangular facets. Phenotypic parameters, including leaf length, leaf width, leaf area, plant height and maximum canopy width, were calculated from reconstructed surfaces. Accurate evaluation in 2D and 3D for individual leaves was performed by comparing reconstructed phenotypic parameters with referenced values and by calculating the Hausdorff distance, i.e. the mean distance between two surfaces. Our analysis demonstrates that there were good agreements in leaf parameters between referenced and estimated values. A high level of overlap was also found between surfaces of image-based reconstructions and laser scanning. Accuracy of 3D reconstruction of curly-leaf plants was relatively lower than that of flat-leaf plants. Plant height of three plants and maximum canopy width of cucumber and pepper showed an increasing trend during the 70 d after transplanting. Maximum canopy width of eggplants reached its peak at the 40th day after transplanting. The larger leaf phenotypic parameters of cucumber were mostly found at the middle-upper leaf position. High-accuracy 3D evaluation of reconstruction quality indicated that dynamic capture of the 3D canopy based on the MVS approach can be potentially used in 3D phenotyping for applications in breeding and field management.

Leaf age is an important trait in the process of maize (Zea mays L.) growth. It is significant to estimate the seed activity and yield of maize by counting leaves. Detection and counting of the maize leaves in the field are very difficult due to the complexity of the field scenes and the cross-covering of adjacent seedling leaves. A method was proposed in this study for detecting and counting maize leaves based on deep learning with RGB images collected by unmanned aerial vehicles (UAVs). The Mask R-CNN was used to separate the complete maize seedlings from the complex background to reduce the impact of weeds on leaf counting. We proposed a new loss function SmoothLR for Mask R-CNN to improve the segmentation performance of the model. Then, YOLOv5 was used to detect and count the individual leaves of maize seedlings after segmentation. The 1005 field seedlings images were randomly divided into the training, validation, and test set with the ratio of 7:2:1. The results showed that the segmentation performance of Mask R-CNN with Resnet50 and SmoothLR was better than that with LI Loss. The average precision of the bounding box (Bbox) and mask (Mask) was 96.9% and 95.2%, respectively. The inference time of single image detection and segmentation was 0.05 s and 0.07 s, respectively. YOLOv5 performed better in leaf detection compared with Faster R-CNN and SSD. YOLOv5x with the largest parameter had the best detection performance. The detection precision of fully unfolded leaves and newly appeared leaves was 92.0% and 68.8%, and the recall rates were 84.4% and 50.0%, respectively. The average precision (AP) was 89.6% and 54.0%, respectively. The rates of counting accuracy for newly appeared leaves and fully unfolded leaves were 75.3% and 72.9%, respectively. The experimental results showed the possibility of current research on exploring leaf counting for field-grown crops based on UAV images.

Accurate segmentation of individual leaves of sugar beet plants is of great significance for obtaining the leaf-related phenotypic data. This paper developed a method to segment the point clouds of sugar beet plants to obtain high-quality segmentation results of individual leaves. Firstly, we used the SFM algorithm to reconstruct the 3D point clouds from multi-view 2D images and obtained the sugar beet plant point clouds after preprocessing. We then segmented them using the multiscale tensor voting method (MSTVM)-based region-growing algorithm, resulting in independent leaves and overlapping leaves. Finally, we used the surface boundary filter (SBF) method to segment overlapping leaves and obtained all leaves of the whole plant. Segmentation results of plants with different complexities of leaf arrangement were evaluated using the manually segmented leaf point clouds as benchmarks. Our results suggested that the proposed method can effectively segment the 3D point cloud of individual leaves for field grown sugar beet plants. The leaf length and leaf area of the segmented leaf point clouds were calculated and compared with observations. The calculated leaf length and leaf area were highly correlated with the observations with R² (0.80–0.82). It was concluded that the MSTVM-based region-growing algorithm combined with SBF can be used as a basic segmentation step for high-throughput plant phenotypic data extraction of field sugar beet plants.

Accurate and real-time field wheat ear counting is of great significance for wheat yield prediction, genetic breeding and optimized planting management. In order to realize wheat ear detection and counting under the large-resolution Unmanned Aerial Vehicle (UAV) video, Space to depth (SPD) module was added to the deep learning model YOLOv7x. The Normalized Gaussian Wasserstein Distance (NWD) Loss function is designed to create a new detection model YOLOv7xSPD. The precision, recall, F1 score and AP of the model on the test set are 95.85%, 94.71%, 95.28%, and 94.99%, respectively. The AP value is 1.67% higher than that of YOLOv7x, and 10.41%, 39.32%, 2.96%, and 0.22% higher than that of Faster RCNN, SSD, YOLOv5s, and YOLOv7. YOLOv7xSPD is combined with the Kalman filter tracking and the Hungarian matching algorithm to establish a wheat ear counting model with the video flow, called YOLOv7xSPD Counter, which can realize real-time counting of wheat ears in the field. In the video with a resolution of 3840×2160, the detection frame rate of YOLOv7xSPD Counter is about 5.5FPS. The counting results are highly correlated with the ground truth number (R^2 = 0.99), and can provide model basis for wheat yield prediction, genetic breeding and optimized planting management.