Radiology Report Generation (RRG) is essential for computer-aided diagnosis and medication guidance, which can relieve the heavy burden of radiologists by automatically generating the corresponding radiology reports according to the given radiology image. However, generating accurate lesion descriptions remains challenging due to spurious correlations from visual-linguistic biases and inherent limitations of radiological imaging, such as low resolution and noise interference. To address these issues, we propose a two-stage framework named Cross-Modal Causal Representation Learning (CMCRL), consisting of the Radiological Cross-modal Alignment and Reconstruction Enhanced (RadCARE) pre-training and the Visual-Linguistic Causal Intervention (VLCI) fine-tuning. In the pre-training stage, RadCARE introduces a degradation-aware masked image restoration strategy tailored for radiological images, which reconstructs high-resolution patches from low-resolution inputs to mitigate noise and detail loss. Combined with a multiway architecture and four adaptive training strategies (e.g., text postfix generation with degraded images and text prefixes), RadCARE establishes robust cross-modal correlations even with incomplete data. In the VLCI phase, we deploy causal front-door intervention through two modules: the Visual Deconfounding Module (VDM) disentangles local-global features without fine-grained annotations, while the Linguistic Deconfounding Module (LDM) eliminates context bias without external terminology databases. Experiments on IU-Xray and MIMIC-CXR show that our CMCRL pipeline significantly outperforms state-of-the-art methods, with ablation studies confirming the necessity of both stages. Code and models are available at.
Physics-informed neural networks (PINN) have shown their potential in solving both direct and inverse problems of partial differential equations. In this paper, we introduce a PINN-based deep learning approach to reconstruct one-dimensional rough surfaces from field data illuminated by an electromagnetic incident wave. In the proposed algorithm, the rough surface is approximated by a neural network, with which the spatial derivatives of surface function can be obtained via automatic differentiation, and then the scattered field can be calculated using the method of moments. The neural network is trained by minimizing the loss between the calculated and the observed field data. Furthermore, the proposed method is an unsupervised approach, independent of any surface data, where only the field data are used. Both transverse electric (TE) field (Dirichlet boundary condition) and transverse magnetic (TM) field (Neumann boundary condition) are considered. Two types of field data are used here: full-scattered field data and phaseless total field data. The performance of the method is verified by testing with Gaussian-correlated random rough surfaces. Numerical results demonstrate that the PINN-based method can recover rough surfaces with great accuracy and is robust with respect to a wide range of problem regimes.
Contrastive learning methods enforce label distance relationships in feature space to improve representation capability for regression models. However, these methods highly depend on label information to correctly recover ordinal relationships of features, limiting their applications to semi-supervised regression. In this work, we extend contrastive regression methods to allow unlabeled data to be used in the semi-supervised setting, thereby reducing the dependence on costly annotations. Particularly we construct the feature similarity matrix with both labeled and unlabeled samples in a mini-batch to reflect inter-sample relationships, and an accurate ordinal ranking of involved unlabeled samples can be recovered through spectral seriation algorithms if the level of error is within certain bounds. The introduction of labeled samples above provides regularization of the ordinal ranking with guidance from the ground-truth label information, making the ranking more reliable. To reduce feature perturbations, we further utilize the dynamic programming algorithm to select robust features for the matrix construction. The recovered ordinal relationship is then used for contrastive learning on unlabeled samples, and we thus allow more data to be used for feature representation learning, thereby achieving more robust results. The ordinal rankings can also be used to supervise predictions on unlabeled samples, serving as an additional training signal. We provide theoretical guarantees and empirical verification through experiments on various datasets, demonstrating that our method can surpass existing state-of-the-art semi-supervised deep regression methods.
Contrastive learning methods enforce label distance relationships in feature space to improve representation capability for regression models. However, these methods highly depend on label information to correctly recover ordinal relationships of features, limiting their applications to semi-supervised regression. In this work, we extend contrastive regression methods to allow unlabeled data to be used in the semi-supervised setting, thereby reducing the dependence on costly annotations. Particularly we construct the feature similarity matrix with both labeled and unlabeled samples in a mini-batch to reflect inter-sample relationships, and an accurate ordinal ranking of involved unlabeled samples can be recovered through spectral seriation algorithms if the level of error is within certain bounds. The introduction of labeled samples above provides regularization of the ordinal ranking with guidance from the ground-truth label information, making the ranking more reliable. To reduce feature perturbations, we further utilize the dynamic programming algorithm to select robust features for the matrix construction. The recovered ordinal relationship is then used for contrastive learning on unlabeled samples, and we thus allow more data to be used for feature representation learning, thereby achieving more robust results. The ordinal rankings can also be used to supervise predictions on unlabeled samples, serving as an additional training signal. We provide theoretical guarantees and empirical verification through experiments on various datasets, demonstrating that our method can surpass existing state-of-the-art semi-supervised deep regression methods. Our code have been released on https://github.com/xmed-lab/CLSS.
1 November 2024 in IEEE Transactions on Computational Imaging
Computed Tomography (CT) has been extensively utilized for medical diagnosis, assessment, as well as treatment planning and guidance. However, the image quality will be significantly compromised when metallic implants are present in patients' bodies, consequently affecting the clinical diagnosis or radiation therapy dose calculation. Previous Metal Artifact Reduction (MAR) methods either require prior knowledge about metallic implants or exhibit modeling bias in the mechanism of artifact formation, which restricts the capability to acquire highquality CT images and increases the complexity of practical applications. In this paper, we propose a novel MAR method based on a feature-to-image conditional normalization flow, named F2IFlow, to address the problem. Specifically, we initially design an inherent feature extraction to get the inherent anatomical features of CT images. Then, a feature-to-image flow module is used for completing the metal-artifact-free CT images progressively through a series of reversible transformations. Incorporating these designs into F2IFlow, the coarse-to-fine strategy equips our model with the capability to deliver exceptional performance. Experimental results on both simulated and clinical datasets demonstrate that our method achieves superior performance in both quantitative and qualitative outcomes, exhibiting better visual effects in terms of artifact reduction and image fidelity.