25 September 2024 in IEEE Transactions on Medical Imaging
Complicated image registration is a key issue in medical image analysis, and deep learning-based methods have achieved better results than traditional methods. The methods include ConvNet-based and Transformer-based methods. Although ConvNets can effectively utilize local information to reduce redundancy via small neighborhood convolution, the limited receptive field results in the inability to capture global dependencies. Transformers can establish long-distance dependencies via a self-attention mechanism; however, the intense calculation of the relationships among all tokens leads to high redundancy. We propose a novel unsupervised image registration method named the unified Transformer and superresolution (UTSRMorph) network, which can enhance feature representation learning in the encoder and generate detailed displacement fields in the decoder to overcome these problems. We first propose a fusion attention block to integrate the advantages of ConvNets and Transformers, which inserts a ConvNet-based channel attention module into a multihead self-attention module. The overlapping attention block, a novel cross-attention method, uses overlapping windows to obtain abundant correlations with match information of a pair of images. Then, the blocks are flexibly stacked into a new powerful encoder. The decoder generation process of a high-resolution deformation displacement field from low-resolution features is considered as a superresolution process. Specifically, the superresolution module was employed to replace interpolation upsampling, which can overcome feature degradation. UTSRMorph was compared to state-of-the-art registration methods in the 3D brain MR (OASIS, IXI) and MR-CT datasets (abdomen, craniomaxillofacial). The qualitative and quantitative results indicate that UTSRMorph achieves relatively better performance. The code and datasets used are publicly available at https://github.com/Runshi-Zhang/UTSRMorph.
29 November 2023 in IEEE Journal of Biomedical and Health Informatics
Craniomaxillofacial (CMF) surgery always relies on accurate preoperative planning to assist surgeons, and automatically generating bone structures and digitizing landmarks for CMF preoperative planning is crucial. Since the soft and hard tissues of the CMF regions possess complicated attachment, segmenting the CMF bones and detecting the CMF landmarks are challenging problems. In this study, we proposed a semantic segmentation network to segment the maxilla, mandible, zygoma, zygomatic arch, and frontal bones. Then, we obtained the minimum bounding box around the CMF bones. After cropping, we used the top-down heatmap landmark detection network, similar to the segmentation module, to identify 18 CMF landmarks from the cropping patch. In addition, an unbiased heatmap encoding method was proposed to generate actual landmark coordinates in the heatmap. To overcome quantization effects in the heatmap-based landmark detection networks, the distribution-prior coordinate representation of medical landmarks (DCRML) was proposed to utilize the prior distribution of the encoding heatmap, approximating the accurate landmark coordinates in heatmap decoding by Taylor's theorem. The encoding and decoding method can easily contribute to other existing landmark detection frameworks based on heatmaps; consequently, these approaches can readily benefit without changing model structure. We used prior segmentation knowledge to enhance the semantic information around the landmarks, increasing landmark detection accuracy. The proposed framework was evaluated by 100 healthy persons and 86 patients from multicenter cooperation. The mean Dice score of our proposed segmentation network achieved over 88 %; in particular, the mandible accuracy was approximately 95%. The mean error of landmarks was 1.84 $\pm$ 1.32 mm.
17 November 2021 in IEEE Sensors Journal
Solder joints of printed circuit board provide the electrical and mechanical connections between electronic component and substrate. A pseudo solder inside the solder joint can reduce its electrical conductivity and often leads to crack initiation and propagation in service. The pseudo solder is a threat to the normal operation of the whole electronic equipment, especially in aerospace crafts. The pseudo solder inside the solder joint is hard to be detected with traditional electrometric method or automated optical inspection. In this paper, a laser pulse stimulated infrared thermography technique is presented to detect and identify the pseudo solder in solder joints. Characteristic parameters of the solder joints were extracted using pulsed phase thermography, temperature change rate, apparent thermal effusivity and higher-order statistics. Two neural network (NN) models based on the input patterns of data were constructed to classify the solder joints. The inputs were composed of seven characteristic parameters obtained or excess temperature evolutions. The NN was trained with the method of batch gradient descent or stochastic gradient descent. The application of principal component analysis (PCA) in data preprocessing and feature extraction was explored. Finally, the optimal scheme for pseudo solder detection with higher recognition rate was discovered. The optimal scheme is that, using PCA to extract the features from the excess temperature evolutions, and then using the stochastic gradient descent to train the neural network. The recognition rate of the sound and pseudo states can reach 95%, while the accuracy of the defective degree classification is approximately 76%.
16 June 2025 in Medical Image Analysis
14 October 2025 in IEEE Transactions on Automation Science and Engineering
Dual-arm robots (e.g., humanoid robots) possess substantial potential for executing collaborative tasks in universal scenarios. However, the workspaces of the individual arms frequently overlap, rendering self-collision avoidance crucial for maintaining safe robotic operations. Existing motion planning-based methods demonstrate inadequate real-time efficacy, and learning-based distance proxy methods are subject to elevated false positive rates. To address these challenges, we present a novel minimum distance prediction neural network for reactive collision avoidance of dual-arm robots, which considers the continuous motion of the robotic arms and the interrelation of joint configurations. The temporal joint configurations are encoded and divided into historical and current features. A state-space model is utilized to capture the temporal dependency of historical features. A self-attention mechanism is employed to model the hidden relationships among current features. The integration of historical and current features via a cross-attention mechanism followed by a gated fusion module allows for precise prediction of the minimum distance between the dual arms. Simulations and real-world experiments, including human-robot interaction and autonomous grasping tasks, were conducted using two redundant robotic arms. The proposed method achieves an average error of 1.804 cm in minimum distance prediction. In dual-arm autonomous grasping experiments, an average error of 1.471 cm is attained. Our approach has improved accuracy by 38.08% over the state-of-the-art methods. No collisions occurred throughout all real-world experiments. This method holds promise for extensive applications of dual-arm robots. The code is accessible at https://github.com/XuejinLuo/SelfCollision.
28 January 2025 in International Journal of Oral and Maxillofacial Surgery