Intelligent unmanned vending machines (UVMs) based on machine vision have attracted great attention in the unmanned retail industry. However, due to the complexity of practical application scenarios and environments, the existing vision-based intelligent UVMs face challenges related to missed-detection and mis-detection of product, and require costly physical components such as the infrared radio frequency sensors to capture shopping behaviors. In this study, we propose a BP-YOLO, the real-time model that integrates optimized YOLOv7 and BlazePose for product detection and shopping behaviors recognition. BP-YOLO can accurately detect the products purchased by consumers and their shopping behaviors in complex scenarios. To address the problems of missed-detection and mis-detection, we introduce the 3D attention mechanism SimAM and the deformable ConvNets v2 (DCNv2) to recombine and optimize the one-stage object detection model YOLOv7. This method reduces the interference of the invalid information in complex scenarios by adaptively weighting each channel and 3D spatial features, focuses on feature information in a sparse space, and minimizes the loss of feature information during the transmission process based on multi-scale feature extraction and fusion. To recognize and judge the shopping behaviors of consumers, we track the hand and arm key points of consumers using the pose estimation model BlazePose. Using the mAP@[0.5:0.95] as the evaluation metric for product detection, the experimental results on a customized product dataset show that BP-YOLO achieves an average accuracy of 96.17% for all product categories detection; the average success rate of consumer shopping recognition reaches 92%, 98%, and 94.7% under three light and noise intensity, respectively. Therefore, our BP-YOLO model for intelligent UVMs has effectiveness in commercial deployment.
Increasing human activities have caused serious disturbance to global forest resources, so how to accurately identify individual trees has become an important task of forest resources investigation. In order to get the accurate number of individual trees, this paper took coniferous forest and mixed coniferous and broad-leaved forest as experimental samples, as well as Digital Orthophoto Map and airborne LiDAR Point Cloud as research data. We propose a deep Learning individual tree segmentation method based on RetinaNet model and PCS algorithm by doing comparative analysis (classical Watershed Algorithm and Layer Stacking Algorithm) at the plots (with high, medium, and low densities). The experimental results show that the method proposed in this paper can solve the problem of individual tree segmentation in high density forest and improve its degree of automation. Compared with Watershed algorithm and Layer stacking algorithm, F-Measure is improved by 6%-29% and 7%-20%, respectively. In other words, the results of individual tree segmentation presented in this paper can not only improve the precision of individual tree segmentation, but also maintain a high detection rate, which can meet the accuracy and high efficiency of individual tree extraction, so as to meet the needs for modern forestry investigation.
29 January 2025 in IEEE Transactions on Geoscience and Remote Sensing
Accurate monitoring of ground subsidence caused by underground coal mining is crucial for environmental protection. This article proposes a correction method for coal mining subsidence, termed Subsidence Correction based on Horizontal Displacement (SCHD), utilizing UAV LiDAR technology. This method leverages the strengths of image sub-pixel correlation technology in obtaining high-precision horizontal displacement information while addressing the limitations of the traditional Digital Elevation Model differential (DoD) method, which often yields incomplete and inaccurate deformation data. Through simulation experiments and field verifications, we quantitatively analyze the effects of surface slope, the angle between the aspect and the horizontal displacement vector, and the magnitude of horizontal displacement on subsidence modeling accuracy. We validate the reliability of the MicMac image sub-pixel correlation technique for extracting horizontal displacement information from the ground surface before and after coal mining, and we correct the subsidence errors introduced by neglecting horizontal displacement using the proposed SCHD method. The results indicate that overlooking horizontal displacement can lead to significant overestimations or underestimations of vertical deformation, with error magnitude closely related to the aforementioned topographic parameters and displacement characteristics. The high-precision horizontal displacement information obtained through the image sub-pixel correlation technique meets subsidence monitoring requirements. The SCHD method effectively reduces subsidence modeling errors and improves monitoring accuracy by 15.6% to 40%, achieving centimeter-level precision. This study underscores the significant impact of horizontal displacement on monitoring coal mining subsidence, demonstrating that the SCHD method enhances the accuracy of such monitoring and provides a robust technical approach for extracting detailed three-dimensional deformation information from mining area surfaces.
The repeated extraction of coal seams in the Loess Plateau mining region has greatly increased the severity of surface deformation and associated damage. Accurately characterizing the spatio-temporal evolution of subsidence and the underlying mechanisms is a critical engineering challenge for mining safety. Taking the Dafosi Coal Mine located in the loess gully region as a case study, this paper thoroughly examines the variations in surface deformation and damage characteristics caused by single and repeated seam mining. The analysis integrates surface movement monitoring data, global navigation satellite system (GNSS) dynamic observations, field surveys, unmanned aerial vehicle (UAV) photogrammetry, and numerical simulation methods. Notably, to ensure the accuracy of prediction parameters, a refined Particle Swarm Optimization (PSO) algorithm incorporating a neighborhood-based mechanism was employed specifically for the inversion of probability integral parameters. The results indicate that the subsidence factor and horizontal movement factor increase markedly following repeated mining. The maximum surface subsidence velocity also increases substantially, and this acceleration remains evident after normalizing by mining thickness and face-advance rate. The fore effective angle is smaller in repeated mining than in single-seam mining, and the duration of surface movement is substantially extended. Repeated mining fractured key strata and caused a functional transition from the classic three-zone response to a two-zone connectivity pattern, while the thick loess cover responds as a disturbed discontinuous-deformation layer, which together aggravates step-like and slope-related damage. The severity of surface damage is strongly influenced by topographic features and geotechnical properties. These findings demonstrate that the proposed integrated approach is highly effective for geological hazard assessment and provides a practical reference for engineering applications in similar complex terrains.
Accurate registration of multi-temporal LiDAR point clouds is essential for reliable monitoring of mining subsidence. Systematic errors in point clouds acquired at different times can arise from GNSS/INS positioning drift, sensor calibration bias, and differences in observation geometry. These errors typically manifest as global reference shifts or gradual distortions. When such errors are superimposed on real terrain changes, they can mask subsidence signals and introduce observational pseudo-differences, thereby increasing the difficulty of separating actual subsidence from artifacts. To address this issue, this study proposes Robust-Registration-Based Systematic Error Correction for Time-Series Point Clouds (RR-SEC), which establishes a consistent reference framework across epochs. The method does not assume that stable areas remain strictly unchanged. Instead, it identifies regions whose local change patterns are more temporally consistent using an information entropy analysis of multi-temporal differences. Under complex terrain, the method selects points with lower difference entropy as stable control points and uses them to constrain the registration process. It then performs Generalized Iterative Closest Point (GICP) rigid registration under these constraints to estimate the overall three-dimensional translation and rotation between point clouds from different periods. The estimated transformation is applied to the entire point cloud to correct inter-epoch reference mismatches and unify the coordinate reference across all epochs. Comprehensive validation using simulated complex terrain data containing rigid reference biases and non-rigid deformations, as well as UAV LiDAR data collected from the MuduChaideng Coal Mine, shows that, compared with the baseline GICP method, RR-SEC reduces alignment errors. It decreases the mean residual in stable areas by approximately 85%. The subsidence values computed from the corrected point clouds are more consistent with measured values, and the spatial deformation patterns are easier to interpret. RR-SEC demonstrates robust performance and can serve as a practical approach to improve the accuracy of deformation monitoring in mining areas and potentially other geoscientific applications.
23 May 2025 in IEEE Transactions on Geoscience and Remote Sensing
Deep learning (DL)-based point cloud segmentation methods have been extensively explored, but the majority focus either on local or global feature learning, with few integrating both. These integrated approaches have not been sufficiently explored in complex mountainous scenes with low feature heterogeneity. To address this gap, we propose a novel point-based multiscale spatial Convolution-Swin Transformer network (Point-SCT). Point-SCT combines convolutional local geometric detail capture with global relationship modeling via dynamic window interactions in Transformer, enhancing ground filtering accuracy in challenging mountainous scenes. The encoder incorporates a convolution-based multiscale local feature aggregation (MLFA) approach, integrating local geometric feature encoding local geometric structure encoding (LGSE) and diluent pooling (DP) strategies to effectively aggregate local detailed geometric features while suppressing irrelevant feature vectors and enhancing the representation of low-heterogeneity feature. Additionally, the dynamic spatial window strategy within the Transformer facilitates the capture of long-range feature dependencies. To mitigate noise introduced by RGB in point cloud overlays and sharpen geometric distinctions between the ground and low-lying vegetation, we introduce boundary detector, curvature, and average elevation (BCE) as prior inputs, replacing RGB. Finally, quantitative and qualitative analyses of Point-SCT are conducted on an airborne laser scanning (ALS) dataset from a mountainous area, with ablation studies validating the effectiveness of LGSE, DP, and BCE. The comprehensive experiments demonstrate that Point-SCT robustly segments ground points in complex mountainous scenes, achieving state-of-the-art levels of accuracy and generalization.