This paper presents a froth image statistical modeling-based online flotation process operation-state identification method by introducing a biologically inspired Gabor wavelet transform in accordance with the physiological findings in the biological vision system. It derived the latent probabilistic density models of these biologically inspired Gabor filtering responses (GFRs) based on a versatile intermediate probability modeling frame, Gaussian scale mixture model. It has demonstrated that both the real and the imaginary representation of GFR obey a Laplace distribution. Accordingly, the amplitude representation of GFR obeys a Gamma distribution. Whereas the phase representation of GFR is an important yet frequently ignored aspect in Gabor-based signal analysis; it is demonstrated to be a periodic distribution and can be expressed by a von Mises-like distribution model. Successively, a local spline regression (LSR)-based classifier that the maps scattered statistical feature points of froth images directly to the operation-state labels smoothly is introduced for the operation-state recognition. Extensive confirmatory and comparative experiments on an industrial-scale bauxite flotation process demonstrate the effectiveness and superiority of the proposed method. Performance effects on different parameter settings, e.g., parameters of Gabor kernel and dimensionalities of multivariate statistical models, are further discussed.

Estimating the 6-D pose of an object is a vital and challenging task for robot vision systems in industrial robotic grasping. With the wide use of 3-D cameras, the additional acquired depth image provides geometric information of the scene to increase the pose estimation performance but leads to a challenge, fully leveraging the two-modal data, the color image and the depth image. Previous works usually adopt two individual strategies to handle the data, which suffer from limited accuracy and efficiency since the two complementary data are not fully explored. Thus, we propose a depth adaptive feature extraction and dense prediction network that decouples the scale-dependent and the scale-invariant information from the depth image. The former guides the network to perceive the 3-D structure of the scene, and the latter, together with color image, provides the scene textures for feature extraction. The proposed network not only fuses multimodal textures but also retains their 3-D structure. In addition, a dense prediction strategy is adopted to regress the object pose; this approach can mitigate the instability caused by outliers. We conduct various evaluations on a real-world industrial dataset to illustrate the advantages of the proposed approach; and a practical robotic grasping platform is presented to demonstrate its application performance.

Accurately estimating the six-degrees of freedom (DoF) pose of unseen objects is crucial for successful robotic manipulation in industrial automation. Some existing methods for this task rely on prior knowledge of individual objects, i.e., the model must be trained on the exact object instance or object category. Others perform unseen object pose estimation but are limited in their feature learning and pose refinement ability. To address these problems, we propose an unseen object pose estimation method that follows a coarse-to-fine framework and leverages the powerful learning ability of diffusion models. We introduce a diffusion model for generating object poses, and conduct a comparison between the generated poses and the original pose to determine the optimal one. We design a novel pose estimation module to provide coarse poses for the PoseDiffusion. This module comprises two feature extraction modules that extract global and masked features. In addition, we propose a strategy to estimate the pose by comparing the similarity between rendered and query poses. The renderings of an unseen object from various viewpoints are generated from its computer-aided design (CAD) model. Our method requires a CAD model of the unseen object only during inference, a scenario well suited to industrial applications. Experimental evaluation on benchmark datasets demonstrates that the proposed framework outperforms existing approaches, achieving state-of-the-art performance in six-DoF object pose estimation.

Visual understanding of 3-D objects is essential for robotic manipulation, autonomous navigation, and augmented reality. However, existing methods struggle to perform this task efficiently and accurately in an end-to-end manner. We propose a single-shot method based on the state space model (SSM) to predict the full 3-D information (pose, size, shape) of multiple 3-D objects from a single RGB-D image in an end-to-end manner. Our method first encodes long-range semantic information from RGB and depth images separately and then combines them into an integrated latent representation that is processed by a modified SSM to infer the full 3-D information in two separate task heads within a unified model. A heatmap/detection head predicts object centers, and a 3-D information head predicts a matrix detailing the pose, size and latent code of shape for each detected object. We also propose a shape autoencoder based on the SSM, which learns canonical shape codes derived from a large database of 3-D point cloud shapes. The end-to-end framework, modified SSM block and SSM-based shape autoencoder form major contributions of this work. Our design includes different scan strategies tailored to different input data representations, such as RGB-D images and point clouds. Extensive evaluations on the REAL275, CAMERA25, and Wild6D datasets show that our method achieves state-of-the-art performance. On the large-scale Wild6D dataset, our model significantly outperforms the nearest competitor, achieving 2.6% and 5.1% improvements on the IOU-50 and 5°10 cm metrics, respectively.

The detection of bubbles in images of TR-PCBs is a difficult task due to the need for precise subregional detection, fast processing speed, and high accuracy. The uneven illumination of the images further complicates the process of bubble segmentation. To address these issues, a TR-PCB bubble segmentation framework, the ECSA-EFA-DT framework, is proposed, which has two stages. In the first stage, an efficient channel and spatial attention (ECSA) module is designed to analyze complex background features by employing the complementary advantages of the channel and spatial position features. Then, an efficient feature-fused autoencoder (EFA) is proposed for region segmentation in TR-PCB images. In the second stage, a dynamic threshold (DT) approach based on maximizing the interclass variance and a sliding window is proposed to eliminate severe shadows and issues with uneven illumination in TR-PCB images. The DT approach segments the bubbles in each region and calculates the bubble rate more accurately than previous methods. The experimental results show that the proposed framework achieves superior performance in several quantitative metrics and is suitable for industrial applications, obtaining an average Dice score of 87.5% and an IoU score of 77.8% on the region segmentation test set, and a 0.312 ME value on the bubble segmentation test set.

In robotic grasp detection, challenges such as uncertainty in object type, size, and placement within the scene diminish grasping accuracy. However, the inability to effectively locate the graspable area and incomplete feature extraction for grasp detection are two key factors that hinder grasp detection accuracy and are not considered in current methods. This paper presents a novel retentive attention-based multiscale perception grasp detection network (RAMPGrasp) to address this constraint. First, we introduce retentive attention in the feature extraction module, which significantly improves the efficiency of attention score computation for long sequences in visual tasks. Second, we propose a multiscale spatial pyramid attention module, which can effectively adjust the importance of multiscale feature sequences and feature channels, while enhancing the correlation of multiscale features. Third, we design the prediction module as a coarse-to-fine framework, improving feature representation for grasp detection by considering the distribution trend of grasp poses. As a result, RAMPGrasp achieves state-of-the-art grasp detection accuracy, with 98.4% and 95.6% on the Cornell and Jacquard datasets, respectively.