Currently, most of the high-performance models for frequency recognition of steady-state visual evoked potentials (SSVEPs) are linear. However, SSVEPs collected from different channels can have non-linear relationship among each other. Linearly combining electroencephalogram (EEG) from multiple channels is not the most accurate solution in SSVEPs classification. To further improve the performance of SSVEP-based braincomputer interface (BCI), we propose a convolutional neural network-based non-linear model, i.e. convolutional correlation analysis (Conv-CA). Different from pure deep learning models, Conv-CA use convolutional neural networks (CNNs) at the top of a self-defined correlation layer. The CNNs function on how to transform multiple channel EEGs into a single EEG signal. The correlation layer calculates the correlation coefficients between the transformed single EEG signal and reference signals. The CNNs provide non-linear operations to combine EEGs in different channels and different time. And the correlation layer constrains the fitting space of the deep learning model. A comparison study between the proposed Conv-CA method and the task-related component analysis (TRCA) based methods is conducted. Both methods are validated on a 40-class SSVEP benchmark dataset recorded from 35 subjects. The study verifies that the Conv-CA method significantly outperforms the TRCA-based methods. Moreover, Conv-CA has good explainability since its inputs of the correlation layer can be analyzed for visualizing what the model learnt from the data. Conv-CA is a non-linear extension of spatial filters. Its CNN structures can be further explored and tuned for reaching a better performance. The structure of combining neural networks and unsupervised features has the potential to be applied to the classification of other signals.
In this paper, we propose a total fractional-order variation model for multiplicative noise removal and contrast enhancement of real SAR images. Inspired by the high dynamic intensity range of SAR images, the full content of the SAR images is preserved by normalizing the original data in this model. Then, we propose a degradation model based on the nonlinear transformation to adjust the intensity of image pixel values. With MAP estimator, a corresponding fidelity term is introduced into the model, which is beneficial for contrast enhancement and bias correction in the denoising process. For the regularization term, a gray level indicator is used as a weighted matrix to make the model adaptive. We first apply the scalar auxiliary variable algorithm to solve the proposed model and prove the convergence of the algorithm. By virtue of the discrete Fourier transform (DFT), the model is solved by an iterative scheme in the frequency domain. Experimental results show that the proposed model can enhance the contrast of natural and SAR images while removing multiplicative noise.
7 August 2023 in IEEE Transactions on Geoscience and Remote Sensing
In recent years, deep learning algorithms for speckle noise removal have attracted much attention. However, speckle noise is strongly heavy-tailed and signal dependent, which makes it difficult to remove. In this paper, we propose a two-stream convolutional neural network with hybrid truncation loss to eliminate multiplicative noise (HTNet). HTNet combines the major task of multiplicative noise removal and the auxiliary task of noise estimation to improve the despeckling effect while preserving texture details. The main branch of HTNet is composed of a feature extraction block and an improved U-Net that can extract multi-scale information, which is mainly used for speckle noise removal. The noise estimation auxiliary branch is designed to fit the speckle noise. A hybrid truncation loss function is to applied for robust estimation of heavy-tailed distribution instead of mean squared error. Extensive experimental results show that HTNet can effectively remove speckle noise and outperforms the state-of-the-art methods on both simulated and real SAR images. In addition, HTNet has advantages on textured images.
21 July 2021 in Journal of Computing and Information Science in Engineering
One of the expectations for the next generation of industrial robots is to work collaboratively with humans as robotic co-workers. Robotic co-workers must be able to communicate with human collaborators intelligently and seamlessly. However, industrial robots in prevalence are not good at understanding human intentions and decisions. We demonstrate a steady-state visual evoked potential (SSVEP)-based brain-computer interface (BCI) which can directly deliver human cognition to robots through a headset. The BCI is applied to a part-picking robot. The BCI sends decisions to the robot while operators visually inspecting the quality of parts. The BCI is verified through a human subject study. In the study, a camera by the side of the conveyor takes photos of each industrial part and presents it to the operator automatically. When the operator looks at the photo, the electroencephalography (EEG) is collected through the BCI. The inspection decision is extracted through SSVEPs in EEG. When a defective part is identified by the operator, the signal is communicated to the robot, which locates the defective part by a second camera and removes it from the conveyor. The robot can grasp various part with our random grasp planning algorithm (2FRG). We have developed a CNN-CCA model for SSVEP extraction. The model is trained on a dataset collected in our offline experiment. Our approach outperforms the existing CCA, CCA-SVM, and PSD-SVM models. The CNN-CCA model is further validated in an online experiment and achieved 93% accuracy in identifying and removing defective parts.
8 June 2024 in Journal of Mathematical Imaging and Vision
Pansharpening, as a widely used technique, plays a crucial role in the field of remote sensing image processing. In this letter, we propose a novel structure tensor-driven block-based variational pansharpening model with adaptive coefficients. First, a structure descriptor derived from the structure tensor of the panchromatic (PAN) image is integrated into the regularization term. It can not only effectively capture the edge information of the PAN image, but also contribute to better preserving the spatial details of the PAN image in the fused product. Furthermore, we partition the degraded PAN image and the upsampled multispectral (MS) image into several equal-sized blocks and utilize a regression-based approach to calculate the adaptive coefficients within each block. As a result, a more accurate constraint relationship between the PAN image and the high-resolution MS image can be established. Then, by incorporating the regularization term and the fidelity term, the proposed variational model is formulated. An explicit finite difference scheme is employed to efficiently solve the gradient descent flow of the proposed model. Experiments conducted on different datasets demonstrate that the proposed pansharpening method outperforms the state-of-the-art techniques in both qualitative and quantitative evaluations.