Ophthalmologists typically require multimodal data sources to improve diagnostic accuracy in clinical decisions. However, due to medical device shortages, low-quality data and data privacy concerns, missing data modalities are common in real-world scenarios. Existing deep learning methods tend to address it by learning an implicit latent subspace representation for different modality combinations. We identify two significant limitations of these methods: (1) implicit representation constraints that hinder the model's ability to capture modality-specific information and (2) modality heterogeneity, causing distribution gaps and redundancy in feature representations. To address these, we propose an Incomplete Modality Disentangled Representation (IMDR) strategy, which disentangles features into explicit independent modal-common and modal-specific features by guidance of mutual information, distilling informative knowledge and enabling it to reconstruct valuable missing semantics and produce robust multimodal representations. Furthermore, we introduce a joint proxy learning module that assists IMDR in eliminating intra-modality redundancy by exploiting the extracted proxies from each class. Experiments on four ophthalmology multimodal datasets demonstrate that the proposed IMDR outperforms the state-of-the-art methods significantly.
As a long-term complication of diabetes, diabetic retinopathy (DR) progresses slowly, potentially taking years to threaten vision. An accurate and robust evaluation of its severity is vital to ensure prompt management and care. Ordinal regression leverages the underlying inherent order between categories to achieve superior performance beyond traditional classification. However, there exist challenges leading to lower DR classification performance: 1) The uneven distribution of DR severity levels, characterized by a long-tailed pattern, adds complexity to the grading process. 2) The ambiguity in defining category boundaries introduces additional challenges, making the classification process more complex and prone to inconsistencies. This work proposes a novel autoregressive ordinal regression method called AOR-DR to address the above challenges by leveraging the clinical knowledge of inherent ordinal information in DR grading dataset settings. Specifically, we decompose the DR grading task into a series of ordered steps by fusing the prediction of the previous steps with extracted image features as conditions for the current prediction step. Additionally, we exploit the diffusion process to facilitate conditional probability modeling, enabling the direct use of continuous global image features for autoregression without relearning contextual information from patch-level features. This ensures the effectiveness of the autoregressive process and leverages the capabilities of pre-trained large-scale foundation models. Extensive experiments were conducted on four large-scale publicly available color fundus datasets, demonstrating our model’s effectiveness and superior performance over six recent state-of-the-art ordinal regression methods. The implementation code is available at https://github.com/Qinkaiyu/AOR-DR.
Unmanned aerial vehicle (UAV) tracking faces significant challenges due to small targets and background interference. Traditional anchor-based tracking algorithms require designing numerous proposals to capture such tiny targets, which entails unacceptable computational overhead. On the other hand, anchor-free tracking methods struggle to adapt to target scale variations, resulting in suboptimal tracking accuracy in anti-UAV tracking scenarios. To address these limitations, we pioneer the integration of diffusion models into visual tracking, proposing SiamDiff—a scale-adaptive anti-UAV framework. We reformulate the tracking task as a bounding box prediction problem, where a diffusion model is leveraged to generate scale-adaptive proposals. Furthermore, we propose a Learnable Mask Module (LMM) and a Frequency Channel Fusion Module (FCFM) to enhance discriminative feature extraction for small targets. Additionally, we design a Scale-Aware Diffusion Strategy (SADA) to boost robustness to scale variations. Experimental results on the Anti-UAV and Anti-UAV410 benchmarks demonstrate the effectiveness of our approach, achieving a State Accuracy (SA) of 71.90% and 67.03%, respectively, outperforming the baseline and other trackers. Moreover, our method shows superior adaptability to scale variations, confirming its robustness in complex anti-UAV tracking scenarios.