Fisheye lens, which is suitable for panoramic imaging, has the prominent advantage of a large field of view and low cost. However, the fisheye image has a severe geometric distortion which may interfere with the stage of image registration and stitching. Aiming to resolve this drawback, we devise a stronger stitching algorithm for fisheye images by combining the traditional image processing method with deep learning. In the stage of fisheye image correction, we propose the Attention-based Nonlinear Activation Free Network (ANAFNet) to deblur fisheye images corrected by Zhang's calibration method. Specifically, ANAFNet adopts the classical single-stage U-shaped architecture based on convolutional neural networks with soft-attention technique and it can restore a sharp image from a blurred image effectively. In the part of image registration, we propose the ORB-FREAK-GMS (OFG), a comprehensive image matching algorithm, to improve the accuracy of image registration. Experimental results demonstrate that panoramic images of superior quality stitching by fisheye images can be obtained through our method.
27 February 2026 in IEEE Transactions on Multimedia
Tooth segmentation is an essential step in digital dental diagnosis and treatment planning, playing a crucial role across various dental specialties. Despite its importance, this process is fraught with challenges due to the high noise and low contrast inherent in 2D and 3D dental radiographic images. Both convolutional neural networks (CNNs) and transformers have shown promise in tooth image segmentation, yet each method has limitations in handling long-range dependencies and computational complexity. To address this issue, this paper introduces T-Mamba, integrating frequency-based features and shared bi-positional encoding into vision mamba to address limitations in modeling global features efficiently. Besides, we designed a gate selection unit to integrate two features in the spatial domain and one feature in the frequency domain adaptively. T-Mamba is the first to integrate frequency-based features into vision mamba, offering the flexibility to process both 2D and 3D tooth image data without the need for separate modules. Additionally, the TED, a large-scale public 2D dental X-ray dataset, has been presented in this paper. Extensive experiments demonstrated that T-Mamba achieved new SOTA performance on both the 3D dental CBCT dataset and our proposed 2D TED dataset. The code and models are publicly available at https://github.com/isbrycee/T-Mamba.
28 January 2025 in IEEE Transactions on Multimedia
Detecting glass regions is a challenging task due to the inherent ambiguity in their transparency and reflective characteristics. Current solutions in this field remain rooted in conventional deep learning paradigms, requiring the construction of annotated datasets and the design of network architectures. However, the evident drawback with these mainstream solutions lies in the time-consuming and labor-intensive process of curating datasets, alongside the increasing complexity of model structures. In this paper, we propose to address these issues by fully harnessing the capabilities of two existing vision foundation models (VFMs): Stable Diffusion and Segment Anything Model (SAM). Firstly, we construct a Synthetic but photorealistic large-scale Glass Surface Detection dataset, dubbed S-GSD, without any labour cost via Stable Diffusion. This dataset consists of four different scales, consisting of 168 k images totally with precise masks. Besides, based on the powerful segmentation ability of SAM, we devise a simple Glass surface sEgMentor named GEM, which follows the simple query-based encoder-decoder architecture. Comprehensive experiments are conducted on the large-scale glass segmentation dataset GSD-S. Our GEM establishes a new state-of-the-art performance with the help of these two VFMs, surpassing the best-reported method GlassSemNet with an IoU improvement of 2.1%. Additionally, extensive experiments demonstrate that our synthetic dataset S-GSD exhibits remarkable performance in zero-shot and transfer learning settings. Codes, datasets, and models are publicly available at: https://github.com/isbrycee/GEM.