The three-dimensional reconstruction method using RGB-D camera has a good balance in hardware cost and point cloud quality. However, due to the limitation of inherent structure and imaging principle, the acquired point cloud has problems such as a lot of noise and difficult registration. This paper proposes a 3D reconstruction method using Azure Kinect to solve these inherent problems. Shoot color images, depth images and near-infrared images of the target from six perspectives by Azure Kinect sensor with black background. Multiply the binarization result of the 8-bit infrared image with the RGB-D image alignment result provided by Microsoft corporation, which can remove ghosting and most of the background noise. A neighborhood extreme filtering method is proposed to filter out the abrupt points in the depth image, by which the floating noise point and most of the outlier noise will be removed before generating the point cloud, and then using the pass-through filter eliminate rest of the outlier noise. An improved method based on the classic iterative closest point (ICP) algorithm is presented to merge multiple-views point clouds. By continuously reducing both the size of the down-sampling grid and the distance threshold between the corresponding points, the point clouds of each view are continuously registered three times, until get the integral color point cloud. Many experiments on rapeseed plants show that the success rate of cloud registration is 92.5% and the point cloud accuracy obtained by this method is 0.789 mm, the time consuming of a integral scanning is 302 s, and with a good color restoration. Compared with a laser scanner, the proposed method has considerable reconstruction accuracy and a significantly ahead of the reconstruction speed, but the hardware cost is much lower when building a automatic scanning system. This research shows a low-cost, high-precision 3D reconstruction technology, which has the potential to be widely used for non-destructive measurement of rapeseed and other crops phenotype.
Rapeseed is an important oil crop in China. Timely estimation of rapeseed stand count at early growth stages provides useful information for precision fertilization, irrigation, and yield prediction. Based on the nature of rapeseed, the number of tillering leaves is strongly related to its growth stages. However, no field study has been reported on estimating rapeseed stand count by the number of leaves recognized with convolutional neural networks (CNNs) in unmanned aerial vehicle (UAV) imagery. The objectives of this study were to provide a case for rapeseed stand counting with reference to the existing knowledge of the number of leaves per plant and to determine the optimal timing for counting after rapeseed emergence at leaf development stages with one to seven leaves. A CNN model was developed to recognize leaves in UAV-based imagery, and rapeseed stand count was estimated with the number of recognized leaves. The performance of leaf detection was compared using sample sizes of 16, 24, 32, 40, and 48 pixels. Leaf overcounting occurred when a leaf was much bigger than others as this bigger leaf was recognized as several smaller leaves. Results showed CNN-based leaf count achieved the best performance at the four- to six-leaf stage with F-scores greater than 90% after calibration with overcounting rate. On average, 806 out of 812 plants were correctly estimated on 53 days after planting (DAP) at the four- to six-leaf stage, which was considered as the optimal observation timing. For the 32-pixel patch size, root mean square error (RMSE) was 9 plants with relative RMSE (rRMSE) of 2.22% on 53 DAP, while the mean RMSE was 12 with mean rRMSE of 2.89% for all patch sizes. A sample size of 32 pixels was suggested to be optimal accounting for balancing performance and efficiency. The results of this study confirmed that it was feasible to estimate rapeseed stand count in field automatically, rapidly, and accurately. This study provided a special perspective in phenotyping and cultivation management for estimating seedling count for crops that have recognizable leaves at their early growth stage, such as soybean and potato.
Background: Verticillium wilt is the major disease of cotton, which would cause serious yield reduction and economic losses, and the identification of cotton verticillium wilt is of great significance to cotton research. However, the traditional method is still manual, which is subjective, inefficient, and labor-intensive, and therefore, this study has proposed a novel method for cotton verticillium wilt identification based on spectral and image feature fusion. The cotton hyper-spectral images have been collected, while the regions of interest (ROI) have been extracted as samples including 499 healthy leaves and 498 diseased leaves, and the average spectral information and RGB image of each sample were obtained. In spectral feature processing, the preprocessing methods including Savitzky-Golay smoothing (SG), multiplicative scatter correction (MSC), de-trending (DT) and mean normalization (MN) algorithms have been adopted, while the feature band extraction methods have adopted principal component analysis (PCA) and successive projections algorithm (SPA). In RGB image feature processing, the EfficientNet was applied to build classification model and 16 image features have been extracted from the last convolutional layer. And then, the obtained spectral and image features were fused, while the classification model was established by support vector machine (SVM) and back propagation neural network (BPNN). Additionally, the spectral full bands and feature bands were used as comparison for SVM and BPNN classification respectively. Result: The results showed that the average accuracy of EfficientNet for cotton verticillium wilt identification was 93.00%. By spectral full bands, SG-MSC-BPNN model obtained the better performance with classification accuracy of 93.78%. By feature bands, SG-MN-SPA-BPNN model obtained the better performance with classification accuracy of 93.78%. By spectral and image fused features, SG-MN-SPA-FF-BPNN model obtained the best performance with classification accuracy of 98.99%. Conclusions: The study demonstrated that it was feasible and effective to use fused spectral and image features based on hyper-spectral imaging to improve identification accuracy of cotton verticillium wilt. The study provided theoretical basis and methods for non-destructive and accurate identification of cotton verticillium wilt.
To improve the treatment strategy of immune‑checkpoint inhibitors for non‑small cell lung cancer (NSCLC), a comprehensive analysis of programmed death‑ligand (PD‑L)1 and PD‑L2 expression is clinically important. The expression of PD‑L1 and PD‑L2 on both tumor cells (TCs) and tumor‑infiltrating immune cells (ICs) was investigated, with respect to tumor‑infiltrating lymphocytes (TILs) and M2 tumor‑associated macrophages (TAMs), which are key components of the tumor microenvironment, in 175 patients with resected NSCLC. The TIL and M2 TAM densities were associated with the expression of PD‑L1 on the two TCs (both P<0.0001) and ICs (both P<0.0001). The TIL and M2 TAM densities were also associated with the expression of PD‑L2 on both TCs (P=0.0494 and P=0.0452, respectively) and ICs (P=0.0048 and P=0.0125, respectively). However, there was no correlation between the percentage of PD‑L1‑positive TCs and the percentage of PD‑L2‑positive TCs (r=0.019; P=0.8049). Meanwhile, tumor differentiation was significantly associated with the PD‑L1 expression on TCs and ICs (P=0.0002 and P<0.0001, respectively). By contrast, tumor differentiation was inversely associated with the PD‑L2 expression on both TCs and ICs (P=0.0260 and P=0.0326, respectively). In conclusion, the combined evaluation of PD‑L1 and PD‑L2 expression could be clinically important in the treatment strategy of immune‑checkpoint inhibitors in patients with NSCLC. In particular, the evaluation of PD‑L2 expression may be necessary for patients with PD‑L1‑negative NSCLC.
Cereals are the main food for mankind. The grain shape extraction and filled/unfilled grain recognition are meaningful for crop breeding and genetic analysis. The conventional measuring method is mainly manual, which is inefficient, labor-intensive and subjective. Therefore, a novel method was proposed to extract the phenotypic traits of cereal grains based on point clouds. First, a structured light scanner was used to obtain the grains point cloud data. Then, the single grain segmentation was accomplished by image preprocessing, plane fitting, region growth clustering. The length, width, thickness, surface area and volume was calculated by the specified analysis algorithms for grain point cloud. To demonstrate this method, experimental materials included rice, wheat and corn were tested. Compared with manual measurement results, the average measurement error of grain length, width and thickness was 2.07%, 0.97%, 1.13%, and the average measurement efficiency was about 9.6 s per grain. In addition, the grain identification model was conducted with 25 grain phenotypic traits, using 6 machine learning methods. The results showed that the best accuracy for filled/unfilled grain classification was 90.184%.The best accuracy for indica and japonica identification was 99.950%, while for different varieties identification was only 47.252%. Therefore, this method was proved to be an efficient and effective way for crop research.
Rice, as a globally vital staple crop, requires efficient field monitoring to ensure optimal growth conditions. This study proposed a novel framework for classifying nutrient deficiencies and formulating fertilization strategies in field-grown rice by fusing UAV-derived vegetation indices (VIs) with deep image features extracted via deep neural networks. The framework integrated visible light VIs, spectral VIs, and image features to provide a comprehensive reflection of crop nutritional conditions, aligning closely with practical production needs. The deep image features achieved nutrition classification accuracies of 88.78% and 84.56% for rice spikelet protection fertilizer application stage (S1) and bud-promoting fertilizer application stage (S2), while the fusion of VIs and deep image features significantly enhanced the accuracy of nutrient classification, with the RF model achieving the highest accuracy (97.50% in S1 and 96.56% in S2). The proposed fertilization strategy effectively improved rice growth traits, demonstrating the potential of UAV-based remote sensing for precision agriculture, which would provide a scalable solution for optimizing rice cultivation and ensuring food security.