The visual feature pyramid has shown its superiority in both effectiveness and efficiency in a variety of applications. However, current methods overly focus on inter-layer feature interactions while disregarding the importance of intra-layer feature regulation. Despite some attempts to learn a compact intra-layer feature representation with the use of attention mechanisms or vision transformers, they overlook the crucial corner regions that are essential for dense prediction tasks. To address this problem, we propose a Centralized Feature Pyramid (CFP) network for object detection, which is based on a globally explicit centralized feature regulation. Specifically, we first propose a spatial explicit visual center scheme, where a lightweight MLP is used to capture the globally long-range dependencies, and a parallel learnable visual center mechanism is used to capture the local corner regions of the input images. Based on this, we then propose a globally centralized regulation for the commonly-used feature pyramid in a top-down fashion, where the explicit visual center information obtained from the deepest intra-layer feature is used to regulate frontal shallow features. Compared to the existing feature pyramids, CFP not only has the ability to capture the global long-range dependencies but also efficiently obtain an all-round yet discriminative feature representation. Experimental results on the challenging MS-COCO validate that our proposed CFP can achieve consistent performance gains on the state-of-the-art YOLOv5 and YOLOX object detection baselines.

RGB-Thermal Salient Object Detection (RGB-T SOD) aims to pinpoint prominent objects within aligned pairs of visible and thermal infrared images. A key challenge lies in bridging the inherent disparities between RGB and Thermal modalities for effective saliency map prediction. Traditional encoder-decoder architectures, while designed for cross-modality feature interactions, may not have adequately considered the robustness against noise originating from defective modalities, thereby leading to suboptimal performance in complex scenarios. Inspired by hierarchical human visual systems, we propose the ConTriNet, a robust Confluent Triple-Flow Network employing a “Divide-and-Conquer” strategy. This framework utilizes a unified encoder with specialized decoders, each addressing different subtasks of exploring modality-specific and modality-complementary information for RGB-T SOD, thereby enhancing the final saliency map prediction. Specifically, ConTriNet comprises three flows: two modality-specific flows explore cues from RGB and Thermal modalities, and a third modality-complementary flow integrates cues from both modalities. ConTriNet presents several notable advantages. It incorporates a Modality-induced Feature Modulator (MFM) in the modality-shared union encoder to minimize inter-modality discrepancies and mitigate the impact of defective samples. Additionally, a foundational Residual Atrous Spatial Pyramid Module (RASPM) in the separated flows enlarges the receptive field, allowing for the capture of multi-scale contextual information. Furthermore, a Modality-aware Dynamic Aggregation Module (MDAM) in the modality-complementary flow dynamically aggregates saliency-related cues from both modality-specific flows. Leveraging the proposed parallel triple-flow framework, we further refine saliency maps derived from different flows through a flow-cooperative fusion strategy , yielding a high-quality, full-resolution saliency map for the final prediction. To evaluate the robustness and stability of our approach, we collect a comprehensive RGB-T SOD benchmark, VT-IMAG , covering various real-world challenging scenarios. Extensive experiments on public benchmarks and our VT-IMAG dataset demonstrate that ConTriNet consistently outperforms state-of-the-art competitors in both common and challenging scenarios, even when dealing with incomplete modality data. The code and VT-IMAG will be available at: https://cser-tang-hao.github.io/contrinet.html .

The purpose of federated learning is to enable multiple clients to jointly train a machine learning model without sharing data. However, the existing methods for training an image segmentation model have been based on an unrealistic assumption that the training set for each local client is annotated in a similar fashion and thus follows the same image supervision level. To relax this assumption, in this work, we propose a label-agnostic unified federated learning framework, named FedMix, for medical image segmentation based on mixed image labels. In FedMix, each client updates the federated model by integrating and effectively making use of all available labeled data ranging from strong pixel-level labels, weak bounding box labels, to weakest image-level class labels. Based on these local models, we further propose an adaptive weight assignment procedure across local clients, where each client learns an aggregation weight during the global model update. Compared to the existing methods, FedMix not only breaks through the constraint of a single level of image supervision but also can dynamically adjust the aggregation weight of each local client, achieving rich yet discriminative feature representations. Experimental results on multiple publicly-available datasets validate that the proposed FedMix outperforms the state-of-the-art methods by a large margin. In addition, we demonstrate through experiments that FedMix is extendable to multi-class medical image segmentation and much more feasible in clinical scenarios. The code is available at: https://github.com/Jwicaksana/FedMix.

The self-attention mechanism has been empirically shown its effectiveness in a wide range of computer vision applications. However, it is usually criticized for the expensive computation cost. Although some revised methods are proposed in the recent past, they are not maturely applicable to remote sensing scene (RSS) images. To address this problem, in this article, we propose a simple yet effective context acquisition module, named thrifty attention, which can capture the long-range dependence efficiently and effectively. Moreover, a recurrent version for thrifty attention, termed recurrent thrifty attention (RTA), is further proposed to take the long-range multihop communications in space–time for RSS images. RTA is a general global contextual information acquisition module that can be used in any hierarchy of deep convolutional neural networks. To demonstrate its superiority, we deploy it to the classical ResNet and establish our proposed RTA Network (RTANet). Extensive experiments are carried out on two levels of the RSS recognition tasks, i.e., the image-level RSS classification and the instance-level RSS object detection. Compared with the standard self-attention mechanism, RTA can reduce at most 0.43 M model parameters while increasing a slight of model floating-point operations per second (FLOPs). Furthermore, results on RSS classification and object detection further verify the accuracy superiority of RTANet.

Medical image segmentation demands the aggregation of global and local feature representations, posing a challenge for current methodologies in handling both long-range and short-range feature interactions. Recently, vision mamba (ViM) models have emerged as promising solutions for addressing model complexities by excelling in long-range feature iterations with linear complexity. However, existing ViM approaches overlook the importance of preserving short-range local dependencies by directly flattening spatial tokens and are constrained by fixed scanning patterns that limit the capture of dynamic spatial context information. To address these challenges, we introduce a simple yet effective method named context clustering ViM (CCViM), which incorporates a context clustering module within the existing ViM models to segment image tokens into distinct windows for adaptable local clustering. Our method effectively combines long-range and short-range feature interactions, thereby enhancing spatial contextual representations for medical image segmentation tasks. Extensive experimental evaluations on diverse public datasets, i.e ., Kumar, CPM17, ISIC17, ISIC18, and Synapse, demonstrate the superior performance of our method compared to current state-of-the-art methods. Our code can be found at https://github.com/zymissy/CCViM.

Image manipulation localization (IML), which seeks to accurately segment tampered regions that are artfully fastened into a normal image, is a fundamental yet challenging computer vision task. Despite that impressive results have been achieved by some progressive deep learning methods, they usually fail in capturing the subtle manipulation artifacts at different object scales, which are not competent to generate a perfect segmentation mask with complete and fine object structures. Besides, the problem of coarse boundaries also occurs frequently. To this end, in this paper, we propose a Transformer-Auxiliary by operator-induced neural Network (TANet) to localize forged regions for IML. Specifically, a stacked multi-scale transformer (SMT) branch is first introduced as a compensation for feature representations of the mainstream convolutional neural network branch. SMT can detect structured abnormalities of the input image at multi-levels by operating on patches of different sizes. Then TANet explicitly exploits an operator induction module (OIM) to excavate valuable and manipulated region-related boundary semantics to guide the representative learning of the mainstream branch. The OIM encourages the network to generate features that highlight object structure, thereby promoting precise boundary localization of forged regions. We conduct extensive experiments on various datasets and settings to validate the effectiveness of TANet. Results show that TANet outperforms the state-of-the-art methods by a large margin under widely-used evaluation metrics.