9 September 2023 in Engineering Applications of Artificial Intelligence
6 January 2026 in Neurocomputing
10 March 2025 in IEEE Transactions on Neural Networks and Learning Systems
Background subtraction in videos is a core challenge in computer vision, aiming to accurately identify moving objects. Robust principal component analysis (RPCA) has emerged as a promising unsupervised (US) paradigm for this task, showing strong performance on various benchmark datasets. Building on RPCA, tensor RPCA (TRPCA) variants have further enhanced background subtraction performance. However, current TRPCA methods often treat moving object pixels independently, lacking spatial-temporal structured-sparsity constraints. This limitation leads to performance degradation in scenarios with dynamic backgrounds, camouflage, and camera jitter. In this work, we introduce a novel spatial-temporal regularized tensor sparse RPCA algorithm to address these issues. By incorporating normalized graph-Laplacian matrices into the sparse component, we enforce spatial-temporal regularization. We construct two graphs—one across spatial locations and another across temporal slices—to guide regularization. By maximizing our objective function, we ensure that the tensor sparse component aligns with the spatiotemporal eigenvectors of the graph-Laplacian matrices, preserving disconnected moving object pixels. We formulate a new objective function and employ batch and online-based optimization methods to jointly optimize background-foreground separation and spatial-temporal regularization. Experimental evaluation on six publicly available datasets demonstrates the superior performance of our algorithm compared to existing methods.
1 January 2026 in IEEE Transactions on Neural Networks and Learning Systems
The preservation of aquatic biodiversity is critical in mitigating the effects of climate change. Aquatic scene understanding plays a pivotal role in aiding marine scientists in their decision-making processes. In this article, we introduce AquaticCLIP, a novel contrastive language-image pretraining (CLIP) model tailored for aquatic scene understanding. AquaticCLIP presents an underwater domain-specific learning framework that aligns images and texts in aquatic environments, enabling tasks such as segmentation, classification, detection, and object counting. By leveraging our large-scale underwater image-text paired dataset without the need for ground-truth (GT) annotations, our model enriches existing vision-language models (VLMs) in the aquatic domain. For this purpose, we construct a 2-million underwater image-text paired dataset using heterogeneous resources, including YouTube, Netflix, National Geographic (NatGeo), etc. To fine-tune AquaticCLIP, we propose a prompt-guided vision encoder (PGVE) that progressively aggregates patch features via learnable prompts, while a vision-guided mechanism enhances the language encoder by incorporating visual context. The model is optimized through a contrastive pretraining loss to align visual and textual modalities. AquaticCLIP achieves notable performance improvements in zero-shot settings across multiple underwater computer vision tasks, outperforming existing methods in both accuracy and robustness. Our model sets a new benchmark for vision-language applications in underwater environments. The code and dataset for AquaticCLIP are publicly available on GitHub at: https://github.com/BasitAlawode/AquaticCLIP.
23 August 2024 in IEEE Access
IGIyyakutti Iyappan Ganapathi
Iyyakutti Iyappan Ganapathi
+1
By improving image quality and resolution, Single Image Super-Resolution (SISR) models help advance understanding of underwater environments. Super-resolution techniques hold promise in addressing these issues by enhancing image details. The transformer architecture has recently gained considerable popularity in low-level vision tasks, including image super-resolution (SR). In this paper, we present a novel multi-distillation algorithm based on the swin transformer and wavelet transform (KDSWT) for underwater image super-resolution. A wavelet block is added to the swin transformer to alleviate the loss of detailed features because wavelet transforms can preserve the high-frequency components, subtle variations, and localized features that contribute to the overall richness in images. Based on a large dataset of high-resolution underwater images, a ViT-based student model is trained to learn the intricate features of underwater images from the teacher model. Using knowledge distillation, the student model learns to upscale input images and capture the domain-specific features essential to understanding underwater scenes. To address the computational cost issue in ViT, we introduce a lightweight network training strategy that combines multi-anchor distillation and progressive learning, enabling the lightweight network to achieve exceptional performance and the fastest inference time compared to recent lightweight state-of-art methods. Our experimental results show that KDSWT lightweight (student) significantly improves peak signal-to-noise ratio (PSNR) over the (original) teacher model while decreasing the number of parameters by 54% and the calculation cost by 22%. Concurrently, KDSWT-SR excels beyond state-of-the-art SR techniques on three benchmark underwater datasets, leading to noteworthy improvements in both PSNR and structural similarity index (SSIM).