In this paper, we design an infrared (IR) and visible (VIS) image fusion via unsupervised dense networks, termed as TPFusion. Activity level measurements and fusion rules are indispensable parts of conventional image fusion methods. However, designing an appropriate fusion process is time-consuming and complicated. In recent years, deep learning-based methods are proposed to handle this problem. However, for multi-modality image fusion, using the same network cannot extract effective feature maps from source images that are obtained by different image sensors. In TPFusion, we can avoid this issue. At first, we extract the textural information of the source images. Then two densely connected networks are trained to fuse textural information and source image, respectively. By this way, we can preserve more textural details in the fused image. Moreover, loss functions we designed to constrain two densely connected convolutional networks are according to the characteristics of textural information and source images. Through our method, the fused image will obtain more textural information of source images. For proving the validity of our method, we implement comparison and ablation experiments from the qualitative and quantitative assessments. The ablation experiments prove the effectiveness of TPFusion. Being compared to existing advanced IR and VIS image fusion methods, our fusion results possess better fusion results in both objective and subjective aspects. To be specific, in qualitative comparisons, our fusion results have better contrast ratio and abundant textural details. In quantitative comparisons, TPFusion outperforms existing representative fusion methods.
30 December 2024 in Neural Computing and Applications
Artificial intelligence-assisted food detection is increasingly moving from single-task classification toward integrated analytical systems capable of producing multiple quality-related outputs from one sensing workflow. However, most hyperspectral food detection studies still rely on a single spectral range or simple feature concatenation, which limits their ability to exploit complementary physicochemical information from heterogeneous sensors. In this study, an artificial intelligence-enabled visible–near-infrared and short-wave infrared (VNIR-SWIR) hyperspectral fusion framework is proposed for multi-task fruit detection, using origin authentication and bruise localization as representative tasks. The proposed method first constructs an observation-consistent fused representation from high-resolution VNIR images and low-resolution SWIR images. Collaborative spectral unmixing is used to couple cross-modal material distributions, while abundance-consistency and downsampled observation-consistency constraints are introduced to estimate SWIR-informed features on the VNIR spatial grid without assuming measured high-resolution SWIR ground truth. The fused representation is then processed by a shared spectral–spatial deep encoder with two task-specific heads: a fruit-level classification head for origin authentication and a pixel-level segmentation head for bruise detection. Experiments on apple and kiwifruit datasets show that the proposed framework outperforms VNIR-only, SWIR-only, bicubic-fusion, CNMF-style fusion, and TV-regularized fusion baselines under five fruit-level stratified random splits. For origin-category authentication, the proposed method achieved an accuracy of almost 93.85 for apples and almost 94.35 for kiwifruit. For bruise localization, the proposed method achieved higher overall accuracy, average accuracy, and Cohen’s kappa across the evaluated fruit categories.
The core volume ratio (CVR) is a key indicator for evaluating the proportion of edible fraction in stone fruits. Traditionally, CVR is determined through destructive sampling by separately measuring the masses of the core and entire fruit. Recently, low-field nuclear magnetic resonance imaging (LF-NMRI) has been introduced as a non-destructive alternative, but its sparse sampling limits the ability to achieve accurate spatial and volumetric quantification of fruit quality. To address this limitation, we propose a novel method for high-precision three-dimensional (3D) modeling of stone fruits. The method acquires tomographic LF-NMRI sequences along three orthogonal axes. Each sequence is segmented into pulp and core regions using a SwinUNet deep learning model and converted into point clouds for each view. Point clouds from the three orthogonal views are registered via a genetic algorithm to align structural information from complementary perspectives and fused into a unified 3D model through Poisson surface reconstruction. Using prunes as a representative case, the method enables accurate quantification of core and entire fruit volumes, achieving a CVR estimation with a mean absolute error of 0.13% compared to manual measurements. The proposed three-view reconstruction strategy yields a volumetric error of only 0.73%, significantly outperforming single-view (4.57%) and dual-view (3.73%) approaches. This technology provides a robust and accurate non-destructive solution for 3D internal quality analysis of fruits.
The blind hyperspectral unmixing of altered minerals holds significant indicative importance for the exploration of sandstone-type uranium deposits. Nevertheless, the distinctive backscattering and multiphase characteristics are typically underestimated by existing methods, resulting in limited accuracy in the unmixing performance of complex mixtures. This study presents a physics-informed Swin Transformer Network (PIST–Net) for the blind hyperspectral unmixing of hematite, goethite, biotite, and chlorite. The proposed model includes a lightweight Swin Transformer encoder for the long-range modeling of spatial and spectral features. Furthermore, a dual-branch decoder with adaptive physical parameters was introduced in the spectral reconstruction section. In this decoder, the backscattering prediction head independently estimates the backscattering factor for each endmember, accounting for the true behavior of the mineral materials. As a key parameter of the Hapke model, the learnable strategy can reduce the spectral error in the single-scattering albedo (SSA) space to improve the accuracy of abundance estimation. The results show that PIST–Net consistently outperformed all six competing models on the altered mineral (AM) dataset and the NASA Reflectance Experiment Laboratory (RELAB) dataset. For mixtures composed of 2–4 endmembers, the mean RMSE of estimated abundances ranges from 0.0312 to 0.1037.