Hyperspectral (HS) pansharpening intends to synthesize a HS image with a registered panchromatic image, to generate an enhanced image with simultaneous high spectral resolution and high spatial resolution. However, the spectral range gap between the two kinds of images and the need to resolve details for many continuous narrow bands make the technique prone to spectral distortion and spatial blurring. To mitigate the problems, we propose a new HS pansharpening framework via spectrally predictive convolutional neural networks (HyperPNN). In our proposed HyperPNN, spectrally predictive structure is introduced to strengthen the spectral prediction capability of a pansharpening network. Following the concept of the proposed HyperPNN, two specific pansharpening convolutional neural network (CNN) models, i.e., HyperPNN1 and HyperPNN2, are designed. Experimental results from three datasets suggest the excellent performance of our CNN-based HS pansharpening methods.

Hyperspectral (HS) pansharpening aims at fusing a low resolution HS (LRHS) image with a panchromatic (PAN) image to obtain a full-resolution HS image. Most of existing HS pansharpening approaches are usually based on traditional multispectral (MS) pansharpening technique, which are not specially tailored for two inherent challenges of the HS pansharpening, i.e., much wider spectral range gap between the two kinds of images and having to recover details in many continuous spectral bands simultaneously. In this paper, we develop new spectral-fidelity convolutional neural networks (called HSpeNets) for HS pansharpening to keep the fidelity of a pansharpened image to its true spectra as much as possible. Our methods particularly focus on the decomposability of HS details, accordingly synthesizing these details progressively, and meanwhile introduce a spectral-fidelity loss. We give theoretical justifications and provide detailed experimental results, showing the superiorities of the proposed HSpeNets with regards to other state-of-the-art pansharpening approaches.

Pansharpening aims at fusing a low-resolution multiband optical (MBO) image, such as a multispectral or a hyperspectral image, with the associated high-resolution panchromatic (PAN) image to yield a high spatial resolution MBO image. Though having achieved superior performances to traditional methods, existing convolutional neural network (CNN)-based pansharpening approaches are still faced with two challenges: alleviating the phenomenon of spectral distortion and improving the interpretation abilities of pansharpening CNNs. In this work, we develop a novel spectral-aware pansharpening neural network (SA-PNN). On the one hand, SA-PNN employs a network structure composed of a detail branch and an approximation branch, which is consistent with the detail injection framework; on the other hand, SA-PNN strengthens processing along the spectral dimension by using a spectral-aware strategy, which involves spatial feature transforms (SFTs) coupling the approximation branch with the detail branch as well as 3D convolution operations in the approximation branch. Our method is evaluated with experiments on real-world multispectral and hyperspectral datasets, verifying its excellent pansharpening performance.

Car sickness, an enormous vehicular travel challenge, affects a significant proportion of the population. Pharmacological interventions are limited by adverse side effects, and effective nonpharmacological alternatives remain to be identified. Here, we introduce a novel attention shifting method based on a closed-loop, artificial intelligence (AI)-driven, wearable mindfulness brain–computer interface (BCI) to alleviate car sickness. As the user performs an attentional task, i.e., focusing on breathing as in mindfulness, with a wearable headband, the BCI collects and analyses electroencephalography (EEG) data via a convolutional neural network to assess the user’s mindfulness state and provide real-time audiovisual feedback. This approach might sustainedly shift the user’s attention from physiological discomfort towards the BCI-based mindfulness practices, thereby mitigating car sickness symptoms. The efficacy of the proposed method was rigorously evaluated in two real-world experiments, namely, short and long car rides, with a large cohort of more than 100 participants susceptible to car sickness. Remarkably, over 83% of the participants rated the BCI-based attention shifting as effective, with significant reductions in car sickness severity, particularly in individuals with severe symptoms. Furthermore, EEG data analysis revealed a neurobiological signature of car sickness, which provided mechanistic insights into the efficacy of the BCI-based attention shifting for alleviating car sickness. This study proposed the first large-scale validated, nonpharmacological and wearable intervention method and system for car sickness, with the potential to transform the travel experiences of hundreds of millions of people suffering from car sickness, which also represents a new application of BCI technology.

Carsickness impairs comfort and affects a large proportion of the population. However, interventions that provide a therapeutic solution to carsickness have yet to be established. Here we introduce a wearable mindfulness meditation brain–computer interface (MM-BCI) system as a closed-loop training therapy for carsickness. The system records electroencephalographic activity, decodes meditative state in real time and delivers audiovisual neurofeedback to scaffold meditation practice. In a 10-week randomized controlled trial, 60 individuals susceptible to carsickness were assigned to practice mindfulness meditation with either real-time MM-BCI neurofeedback or sham feedback, both during real-world car riding and at home. Critically, pre-intervention, post-intervention, and one-month follow-up assessments of carsickness severity were conducted during regular car riding without any task or feedback system. Relative to the sham group, the MM-BCI group showed significantly reduced carsickness severity at post-intervention and follow-up. At baseline, carsickness-susceptible participants exhibited a reduced aperiodic exponent in occipito-parietal cortex relative to non-susceptible controls, identifying a candidate neural signature of carsickness susceptibility. MM-BCI training increased this exponent toward non-susceptible levels, and the magnitude of this neural normalization was associated with the degree of symptom improvement. This study provides the first demonstration that BCI-enhanced mindfulness meditation can induce promising treatment effect on carsickness, offering a transformative non-pharmacological approach to enhance passenger well-being in everyday transit.