Sensor-based sorters operating in the Near-Infrared (NIR) range are commonly used to sort post-consumer plastics. However, this method fails, when the NIR light is fully absorbed by carbon black pigments, which are present in black plastics. Mid-Wave Infrared (MWIR) is less absorbed by carbon black and therefore provides a promising alternative wavelength range for analyzing black polymers. This study compares MWIR to NIR hyperspectral imaging for classifying black and colored plastic waste. We collected spectral data from five common polymers found in post-consumer packaging, namely High-Density Polyethylene (HDPE), Low-Density Polyethylene (LDPE), Polyethylene Terephthalate (PET), Polypropylene (PP), and Polystyrene (PS). We classified the spectra using several chemometric methods, including a convolutional neural network (CNN). The results quantitatively verify the superior performance of MWIR over NIR for classifying black plastics. MWIR achieved a balanced accuracy of 83.4% compared to 47.5% for NIR, when using a CNN, which outperformed other chemometric methods for all sensors and sample sets. On the other hand, NIR surpasses MWIR by 7 percentage points for colored plastics in balanced accuracy. These findings suggest that MWIR hyperspectral imaging is an effective alternative to NIR hyperspectral imaging for sorting post-consumer packaging waste, especially when the share of black plastics is high.

This study presents an annotated multi-sensor, multimodal, and hyperspectral dataset designed to support deep learning-based classification and segmentation of bulky waste. The dataset comprises four distinct sensor modalities: high-resolution visible RGB images (VIS), hyperspectral near-infrared (NIR), temporally resolved thermal infrared (IR), and terahertz (THz) imaging with depth information, providing complementary multimodal information. An image registration process aligns all modalities to a common reference frame, enabling near pixel-precise fusion across sensors. WoodVIT contains 56 registered multi-sensor scenes, partitioned into 22,659 annotated patches with two main classes (wood and non-wood) and 16 subclass labels. It includes pixel-masks and patch-wise annotations to facilitate both segmentation and classification tasks. The primary benchmark task is binary discrimination of wood versus non-wood. The dataset also includes challenging scenarios involving occlusion and concealed contaminants (e.g., embedded metals) to motivate robust multimodal fusion approaches. We provide predefined train/validation/test splits and report baseline results using convolutional neural networks and fusion architectures to establish reference performance. WoodVIT is publicly available to support research on multi-sensor learning for waste sorting.

Multimodal semantic segmentation with heterogeneous sensors must reconcile complementary information across modalities that differ in spatial resolution and channel dimensionality. In particular, high-resolution RGB imaging provides detailed spatial structure but often fails to distinguish visually similar materials, whereas hyperspectral imaging (HSI) provides discriminative spectral signatures but at lower spatial resolution. We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. Although our evaluation targets RGB-HSI fusion,BCAF is modality-agnostic and applies to co-registered RGB with lower-resolution, high-channel auxiliary sensors. On the benchmark SpectralWaste dataset, BCAF delivers strong performance, achieving 75.4% at 55 images/s. We further evaluate a novel industrial dataset: K3I-Cycling (first RGB subset already released on Fordatis). On this dataset, BCAF reaches 62.3% mIoU for material segmentation (paper, metal, plastic, etc.) and 66.2% mIoU for plastic-type segmentation (PET, PP, HDPE, LDPE, PS, etc.). These results show that preserving native-grid spatial detail and spectral structure improves multimodal segmentation under real-time constraints. Code and model checkpoints are publicly available at https://github.com/jonasvilhofunk/BCAF_2026.

Conventional sensor-based sorting systems often rely on handcrafted image-processing features, which are highly task-specific, require expert knowledge, and are difficult to transfer to new sorting tasks. In contrast, deep learning enables automated feature learning and has previously been shown to outperform color-based sorting in several application scenarios. In this study, we investigate the suitability of two deep-learning-based semantic segmentation architectures, U-Net and SegFormer, for real-time sensor-based sorting. Their performance is evaluated in both offline experiments and practical sorting trials for the applications bulky waste sorting and hibiscus tea sorting. Both models proved suitable for the investigated tasks. U-Net showed slight advantages in inference speed and purity, reaching purities of 95.9 wt.% for bulky waste and 100.0 wt.% for hibiscus tea. SegFormer achieved almost equal results for sorting hibiscus tea and four percentage points lower purity for sorting bulky waste. Real-time applicability was demonstrated for SegFormer variants B0 to B3, which met the required throughput of 62 megapixels per second. Overall, the results show that transformer-based models are a viable and promising alternative to CNNs for real-time sensor-based sorting.

Growing waste streams and the transition to a circular economy require efficient automated waste sorting. In industrial settings, materials move on fast conveyor belts, where reliable identification and ejection demand pixel-accurate segmentation. RGB imaging delivers high-resolution spatial detail, which is essential for accurate segmentation, but it confuses materials that look similar in the visible spectrum. Hyperspectral imaging (HSI) provides spectral signatures that separate such materials, yet its lower spatial resolution limits detail. Effective waste sorting therefore needs methods that fuse both modalities to exploit their complementary strengths. We present Bidirectional Cross-Attention Fusion (BCAF), which aligns high-resolution RGB with low-resolution HSI at their native grids via localized, bidirectional cross-attention, avoiding pre-upsampling or early spectral collapse. BCAF uses two independent backbones: a standard Swin Transformer for RGB and an HSI-adapted Swin backbone that preserves spectral structure through 3D tokenization with spectral self-attention. We also analyze trade-offs between RGB input resolution and the number of HSI spectral slices. Although our evaluation targets RGB-HSI fusion, BCAF is modality-agnostic and applies to co-registered RGB with lower-resolution, high-channel auxiliary sensors. On the benchmark SpectralWaste dataset, BCAF achieves state-of-the-art performance of 76.4\% mIoU at 31 images/s and 75.4\% mIoU at 55 images/s. We further evaluate a novel industrial dataset: K3I-Cycling (first RGB subset already released on Fordatis). On this dataset, BCAF reaches 62.3\% mIoU for material segmentation (paper, metal, plastic, etc.) and 66.2\% mIoU for plastic-type segmentation (PET, PP, HDPE, LDPE, PS, etc.).