Xiang Zhang

Xiang Zhang

Auto-generated profile, based on publicly available information

Xiang Zhang

Department of Electronic and Computer Engineering, The Hong Kong University of Science and Technology, Hong Kong SAR, China

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
9 August 2019 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Verified
Journal Article
Closed access
XZXiang Zhang
Xiang Zhang
CLCamilo Libedinsky
Camilo Libedinsky
RSRosa Qiyue So
Rosa Qiyue So
Jose PrincipeJose Principe
Jose Principe
Yiwen WangYiwen Wang
Yiwen Wang
Neuroprosthesis enables the brain control on the external devices purely using neural activity for paralyzed people. Supervised learning decoders recalibrate or re-fit the discrepancy between the desired target and decoder’s output, where the correction may over-dominate the user’s intention. Reinforcement learning decoder allows users to actively adjust their brain patterns through trial and error, which better represents the subject’s motive. The computational challenge is to quickly establish new state-action mapping before the subject becomes frustrated. Recently proposed quantized attention-gated kernel reinforcement learning (QAGKRL) explores the optimal nonlinear neural-action mapping in the Reproducing Kernel Hilbert Space (RKHS). However, considering all past data in RKHS is less efficient and sensitive to detect the new neural patterns emerging in brain control. In this paper, we propose a clustering-based kernel RL algorithm. New neural patterns emerge and are clustered to represent the novel knowledge in brain control. The current neural data only activate the nearest subspace in RKHS for more efficient decoding. The dynamic clustering makes our algorithm more sensitive to new brain patterns. We test our algorithm on both the synthetic and real-world spike data. Compared with QAGKRL, our algorithm can achieve a quicker knowledge adaptation in brain control with less computational complexity.

24 November 2020 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Verified
Journal Article
Closed access
XSXiang Shen
Xiang Shen
XZXiang Zhang
Xiang Zhang
YHYifan Huang
Yifan Huang
SCShuhang Chen
Shuhang Chen
Yiwen WangYiwen Wang
Yiwen Wang
Autonomous brain machine interfaces (BMIs) aim to enable paralyzed people to self-evaluate their movement intention to control external devices. Previous reinforcement learning (RL)-based decoders interpret the mapping between neural activity and movements using the external reward for well-trained subjects, and have not investigated the task learning procedure. The brain has developed a learning mechanism to identify the correct actions that lead to rewards in the new task. This internal guidance can be utilized to replace the external reference to advance BMIs as an autonomous system. In this study, we propose to build an internally rewarded reinforcement learning-based BMI framework using the multi-site recording to demonstrate the autonomous learning ability of the BMI decoder on the new task. We test the model on the neural data collected over multiple days while the rats were learning a new lever discrimination task. The primary motor cortex (M1) and medial prefrontal cortex (mPFC) spikes are interpreted by the proposed RL framework into the discrete lever press actions. The neural activity of the mPFC post the action duration is interpreted as the internal reward information, where a support vector machine is implemented to classify the reward vs. non-reward trials with a high accuracy of 87.5% across subjects. This internal reward is used to replace the external water reward to update the decoder, which is able to adapt to the nonstationary neural activity during subject learning. The multi-cortical recording allows us to take in more cortical recordings as input and uses internal critics to guide the decoder learning. Comparing with the classic decoder using M1 activity as the only input and external guidance, the proposed system with multi-cortical recordings shows a better decoding accuracy. More importantly, our internally rewarded decoder demonstrates the autonomous learning ability on the new task as the decoder successfully addresses the time-variant neural patterns while subjects are learning, and works asymptotically as the subjects’ behavioral learning progresses. It reveals the potential of endowing BMIs with autonomous task learning ability in the RL framework.

1 October 2023 in Journal of Neural Engineering
Verified
Journal Article
Closed access
JTJieyuan Tan
Jieyuan Tan
XZXiang Zhang
Xiang Zhang
SWShenghui Wu
Shenghui Wu
ZSZhiwei Song
Zhiwei Song
SCShuhang Chen
Shuhang Chen
Yiwen WangYiwen Wang
Yiwen Wang
Objectives. Coadaptive brain–machine interfaces (BMIs) allow subjects and external devices to adapt to each other during the closed-loop control, which provides a promising solution for paralyzed individuals. Previous studies have focused on either improving sensory feedback to facilitate subject learning or developing adaptive algorithms to maintain stable decoder performance. In this work, we aim to design an efficient coadaptive BMI framework which not only facilitates the learning of subjects on new tasks with designed sensory feedback, but also improves decoders’ learning ability by extracting sensory feedback-induced evaluation information. Approach. We designed dynamic audio feedback during the trial according to the subjects’ performance when they were trained to learn a new behavioral task. We compared the learning performance of two groups of Sprague Dawley rats, one with and the other without the designed audio feedback to show whether this audio feedback could facilitate the subjects’ learning. Compared with the traditional closed-loop in BMI systems, an additional closed-loop involving medial prefrontal cortex (mPFC) activity was introduced into the coadaptive framework. The neural dynamics of audio-induced mPFC activity was analyzed to investigate whether a significant neural response could be triggered. This audio-induced response was then translated into reward expectation information to guide the learning of decoders on a new task. The multiday decoding performance of the decoders with and without audio-induced reward expectation was compared to investigate whether the extracted information could accelerate decoders to learn a new task. Main results. The behavior performance comparison showed that the average days for rats to achieve 80% well-trained behavioral performance was improved by 26.4% after introducing the designed audio feedback sequence. The analysis of neural dynamics showed that a significant neural response of mPFC activity could be elicited by the audio feedback and the visualization of audio-induced neural patterns was emerged and accompanied by the behavioral improvement of subjects. The multiday decoding performance comparison showed that the decoder taking the reward expectation information could achieve faster task learning by 33.8% on average across subjects. Significance. This study demonstrates that the designed audio feedback could improve the learning of subjects and the mPFC activity induced by audio feedback can be utilized to improve the decoder’s learning efficiency on new tasks. The coadaptive framework involving mPFC dynamics in the closed-loop interaction can advance the BMIs into a more adaptive and efficient system with learning ability on new tasks.

11 October 2022 in IEEE Transactions on Neural Systems and Rehabilitation Engineering
Verified
Journal Article
Open access
XSXiang Shen
Xiang Shen
XZXiang Zhang
Xiang Zhang
YHYifan Huang
Yifan Huang
SCShuhang Chen
Shuhang Chen
ZYZhuliang Yu
Zhuliang Yu
Yiwen WangYiwen Wang
Yiwen Wang
Reinforcement-learning (RL)-based brain-machine interfaces (BMIs) interpret dynamic neural activity into movement intention without patients’ real limb movements, which is promising for clinical applications. A movement task generally requires the subjects to reach the target within one step and rewards the subjects instantaneously. However, a real BMI scenario involves tasks that require multiple steps, during which sensory feedback is provided to indicate the status of the prosthesis, and the reward is only given at the end of the trial. Actually, subjects internally evaluate the sensory feedback to adjust motor activity. Existing RL-BMI tasks have not fully utilized the internal evaluation from the brain upon the sensory feedback to guide the decoder training, and there lacks an effective tool to assign credit for the multi-step decoding task. We propose first to extract intermediate guidance from the medial prefrontal cortex (mPFC) to assist the learning of multi-step decoding in an RL framework. To effectively explore the neural-action mapping in a large state-action space, a temporal difference (TD) method is incorporated into quantized attention-gated kernel reinforcement learning (QAGKRL) to assign the credit over the temporal sequence of movement, but also discriminate spatially in the Reproducing Kernel Hilbert Space (RKHS). We test our approach on the data collected from the primary motor cortex (M1) and the mPFC of rats when they brain control the cursor to reach the target within multiple steps. Compared with the models which only utilize the final reward, the intermediate evaluation interpreted from the mPFC can help improve the prediction accuracy by 10.9% on average across subjects, with faster convergence and more stability. Moreover, our proposed algorithm further increases 18.2% decoding accuracy compared with existing TD-RL methods. The results reveal the possibility of achieving better multi-step decoding performance for more complicated BMI tasks.

29 July 2022 in Journal of Neural Engineering
Verified
Journal Article
Closed access
SWShenghui Wu
Shenghui Wu
CQCunle Qian
Cunle Qian
XSXiang Shen
Xiang Shen
XZXiang Zhang
Xiang Zhang
YHYifan Huang
Yifan Huang
Yiwen WangYiwen Wang
Yiwen Wang
Objectives . Brain–machine interfaces (BMIs) aim to help people with motor disabilities by interpreting brain signals into motor intentions using advanced signal processing methods. Currently, BMI users require intensive training to perform a pre-defined task, not to mention learning a new task. Thus, it is essential to understand neural information pathways among the cortical areas in task learning to provide principles for designing BMIs with learning abilities. We propose to investigate the relationship between the medial prefrontal cortex (mPFC) and primary motor cortex (M1), which are actively involved in motor control and task learning, and show how information is conveyed in spikes between the two regions on a single-trial basis by computational models. Approach . We are interested in modeling the functional relationship between mPFC and M1 activities during task learning. Six Sprague Dawley rats were trained to learn a new behavioral task. Neural spike data was recorded from mPFC and M1 during learning. We then implement the generalized linear model, the second-order generalized Laguerre–Volterra model, and the staged point-process model to predict M1 spikes from mPFC spikes across multiple days during task learning. The prediction performance is compared across different models or learning stages to reveal the relationship between mPFC and M1 spike activities. Main results . We find that M1 neural spikes can be well predicted from mPFC spikes on the single-trial level, which indicates a highly correlated relationship between mPFC and M1 activities during task learning. By comparing the performance across models, we find that models with higher nonlinear capacity perform significantly better than linear models. This indicates that predicting M1 activity from mPFC activity requires the model to consider higher-order nonlinear interactions beyond pairwise interactions. We also find that the correlation coefficient between the mPFC and M1 spikes increases during task learning. The spike prediction models perform the best when the subjects become well trained on the new task compared with the early and middle stages. The results suggest that the co-activation between mPFC and M1 activities evolves during task learning, and becomes stronger as subjects become well trained. Significance . This study demonstrates that the dynamic patterns of M1 spikes can be predicted from mPFC spikes during task learning, and this will further help in the design of adaptive BMI decoders for task learning.

12 June 2021 in Entropy
Verified
Journal Article
Open access
PDF file
XLXi Liu
Xi Liu
SCShuhang Chen
Shuhang Chen
XSXiang Shen
Xiang Shen
XZXiang Zhang
Xiang Zhang
Yiwen WangYiwen Wang
Yiwen Wang
Neural signal decoding is a critical technology in brain machine interface (BMI) to interpret movement intention from multi-neural activity collected from paralyzed patients. As a commonly-used decoding algorithm, the Kalman filter is often applied to derive the movement states from high-dimensional neural firing observation. However, its performance is limited and less effective for noisy nonlinear neural systems with high-dimensional measurements. In this paper, we propose a nonlinear maximum correntropy information filter, aiming at better state estimation in the filtering process for a noisy high-dimensional measurement system. We reconstruct the measurement model between the high-dimensional measurements and low-dimensional states using the neural network, and derive the state estimation using the correntropy criterion to cope with the non-Gaussian noise and eliminate large initial uncertainty. Moreover, analyses of convergence and robustness are given. The effectiveness of the proposed algorithm is evaluated by applying it on multiple segments of neural spiking data from two rats to interpret the movement states when the subjects perform a two-lever discrimination task. Our results demonstrate better and more robust state estimation performance when compared with other filters.