Xiang Gao

Xiang Gao

Auto-generated profile, based on publicly available information

Xiang Gao

Jihua Laboratory, Foshan 528200, China

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
16 January 2019 in IEEE Transactions on Cybernetics
Verified
Journal Article
Closed access
YWYue Wen
Yue Wen
JSJennie Si
Jennie Si
ABAndrea Brandt
Andrea Brandt
XGXiang Gao
Xiang Gao
HHHe Helen Huang
He Helen Huang
Robotic prostheses deliver greater function than passive prostheses, but we face the challenge of tuning a large number of control parameters in order to personalize the device for individual amputee users. This problem is not easily solved by traditional control designs or the latest robotic technology. Reinforcement learning (RL) is naturally appealing. The recent, unprecedented success of AlphaZero demonstrated RL as a feasible, large-scale problem solver. However, the prosthesis-tuning problem is associated with several unaddressed issues such as that it does not have a known and stable model, the continuous states and controls of the problem may result in a curse of dimensionality, and the human-prosthesis system is constantly subject to measurement noise, environmental change and human-body-caused variations. In this paper, we demonstrated the feasibility of direct heuristic dynamic programming, an approximate dynamic programming (ADP) approach, to automatically tune the 12 robotic knee prosthesis parameters to meet individual human users’ needs. We tested the ADP-tuner on two subjects (one able-bodied subject and one amputee subject) walking at a fixed speed on a treadmill. The ADP-tuner learned to reach target gait kinematics in an average of 300 gait cycles or 10 min of walking. We observed improved ADP tuning performance when we transferred a previously learned ADP controller to a new learning session with the same subject. To the best of our knowledge, our approach to personalize robotic prostheses is the first implementation of online ADP learning control to a clinical problem involving human subjects.

27 May 2021 in IEEE Transactions on Robotics
Verified
Journal Article
Closed access
MLMinhan Li
Minhan Li
YWYue Wen
Yue Wen
XGXiang Gao
Xiang Gao
JSJennie Si
Jennie Si
HHHe Huang
He Huang
Personalizing medical devices such as lower limb wearable robots is challenging. While the initial feasibility of automating the process of knee prosthesis control parameter tuning has been demonstrated in a principled way, the next critical issue is to improve tuning efficiency and speed it up for the human user, in clinic settings, while maintaining human safety. We, therefore, propose a policy iteration with constraint embedded (PICE) method as an innovative solution to the problem under the framework of reinforcement learning. Central to PICE is the use of a projected Bellman equation with a constraint of assuring positive semidefiniteness of performance values during policy evaluation. Additionally, we developed both online and offline PICE implementations that provide additional flexibility for the designer to fully utilize measurement data, either from on-policy or off-policy, to further improve PICE tuning efficiency. Our human subject testing showed that the PICE provided effective policies with significantly reduced tuning time. For the first time, we also experimentally evaluated and demonstrated the robustness of the deployed policies by applying them to different tasks and users. Putting it together, our new way of problem solving has been effective as PICE has demonstrated its potential toward truly automating the process of control parameter tuning for robotic knee prosthesis users.

6 May 2021 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
XGXiang Gao
Xiang Gao
JSJennie Si
Jennie Si
YWYue Wen
Yue Wen
MLMinhan Li
Minhan Li
HHHe Huang
He Huang
We are motivated by the real challenges presented in a human–robot system to develop new designs that are efficient at data level and with performance guarantees, such as stability and optimality at system level. Existing approximate/adaptive dynamic programming (ADP) results that consider system performance theoretically are not readily providing practically useful learning control algorithms for this problem, and reinforcement learning (RL) algorithms that address the issue of data efficiency usually do not have performance guarantees for the controlled system. This study fills these important voids by introducing innovative features to the policy iteration algorithm. We introduce flexible policy iteration (FPI), which can flexibly and organically integrate experience replay and supplemental values from prior experience into the RL controller. We show system-level performances, including convergence of the approximate value function, (sub)optimality of the solution, and stability of the system. We demonstrate the effectiveness of the FPI via realistic simulations of the human–robot system. It is noted that the problem we face in this study may be difficult to address by design methods based on classical control theory as it is nearly impossible to obtain a customized mathematical model of a human–robot system either online or offline. The results we have obtained also indicate the great potential of RL control to solving realistic and challenging problems with high-dimensional control inputs.

24 February 2023 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
XGXiang Gao
Xiang Gao
JSJennie Si
Jennie Si
HHHe Huang
He Huang
We aim at creating a transfer reinforcement learning framework that allows the design of learning controllers to leverage prior knowledge extracted from previously learned tasks and previous data to improve the learning performance of new tasks. Toward this goal, we formalize knowledge transfer by expressing knowledge in the value function in our problem construct, which is referred to as reinforcement learning with knowledge shaping (RL-KS). Unlike most transfer learning studies that are empirical in nature, our results include not only simulation verifications but also an analysis of algorithm convergence and solution optimality. Also different from the well-established potential-based reward shaping methods which are built on proofs of policy invariance, our RL-KS approach allows us to advance toward a new theoretical result on positive knowledge transfer. Furthermore, our contributions include two principled ways that cover a range of realization schemes to represent prior knowledge in RL-KS. We provide extensive and systematic evaluations of the proposed RL-KS method. The evaluation environments not only include classical RL benchmark problems but also include a challenging task of real-time control of a robotic lower limb with a human user in the loop.

25 January 2022 in IEEE Robotics and Automation Letters
Verified
Journal Article
Closed access
ZHZiye Hu
Ziye Hu
WLWei Li
Wei Li
ZGZhongxue Gan
Zhongxue Gan
WGWeikun Guo
Weikun Guo
JZJiwei Zhu
Jiwei Zhu
DZDecheng Zhou
Decheng Zhou
Although robots have been widely applied in various fields, allowing a robot to perform a wide range of tasks like humans is a significant challenge. One promising method is meta-learning, which enables robots to learn from demonstrations with the concept of “learning to learn.” However, most meta-learning methods only focus on teaching robots to learn from a single demonstration domain, i.e., demonstrations recording videos of a human or robot performing tasks (human or robot demonstrations). Given that humans can acquire and merge knowledge from various related domains, this letter proposes a novel yet efficient Random Domain-Adaptive Meta-Learning (RDAML) framework to teach the robot to learn from multiple demonstration domains (e.g., human demonstrations + robot demonstrations) with different random sampling parameters. Once training is complete, the trained model can adapt to new environments given a corresponding visual demonstration. Extensive experimental results show that the model trained by our proposed RDAML algorithm obtains better generalization capability. We have demonstrated the effectiveness of our RDAML on real-world placing experiments using a UR5 robot arm, which significantly outperforms current state-of-the-art methods using either human demonstrations or robot demonstrations to teach the robot during testing.

6 February 2025 in IEEE Signal Processing Letters
Verified
Journal Article
Closed access
YCYinqi Chen
Yinqi Chen
PLPeiwen Li
Peiwen Li
YZYangting Zheng
Yangting Zheng
WLWeijian Luo
Weijian Luo
XGXiang Gao
Xiang Gao
The existing deep hashing methods for image retrieval typically modeling hash-coding layer in real number space. However, these methods frequently overlook the intrinsic information loss that occurs during the hash-coding process, as the hash layer performs two tasks simultaneously: spatial transformation and dimensionality reduction. Especially in multi-label image retrieval, the exponential increase in the number of combinations with the labels further amplifies the information loss in Hamming space. Consequently, the efficiency of the hash-coding is unsatisfactory. To mitigate this limitation, we introduce a novel approach termed WalshHash, which is grounded in the principles of Walsh transformation in signal processing. Unlike conventional techniques, WalshHash formulates the hash-coding layer as a filtering process based on Kolmogorov-Arnold Networks (KANs) in the Walsh domain accompanied by constraint loss functions on multiple domains. It ensures the dimensionality reduction in the Walsh domain can be effectively projected onto the real number domain with minimal information loss, because the Walsh space encapsulates the critical information components. As a result, WalshHash demonstrates superior performance in multi-label image retrieval compared to state-of-the-art (SOTA) methods.