Zixu Zhao

Zixu Zhao

Auto-generated profile, based on publicly available information

Zixu Zhao

Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
29 March 2021 in IEEE Transactions on Medical Imaging
Verified
Journal Article
Closed access
YJYueming Jin
Yueming Jin
YLYonghao Long
Yonghao Long
CCCheng Chen
Cheng Chen
ZZZixu Zhao
Zixu Zhao
QDQi Dou
Qi Dou
PHPheng-Ann Heng
Pheng-Ann Heng
Automatic surgical workflow recognition is a key component for developing context-aware computer-assisted systems in the operating theatre. Previous works either jointly modeled the spatial features with short fixed-range temporal information, or separately learned visual and long temporal cues. In this paper, we propose a novel end-to-end temporal memory relation network (TMRNet) for relating long-range and multi-scale temporal patterns to augment the present features. We establish a long-range memory bank to serve as a memory cell storing the rich supportive information. Through our designed temporal variation layer, the supportive cues are further enhanced by multi-scale temporal-only convolutions. To effectively incorporate the two types of cues without disturbing the joint learning of spatio-temporal features, we introduce a non-local bank operator to attentively relate the past to the present. In this regard, our TMRNet enables the current feature to view the long-range temporal dependency, as well as tolerate complex temporal extents. We have extensively validated our approach on two benchmark surgical video datasets, M2CAI challenge dataset and Cholec80 dataset. Experimental results demonstrate the outstanding performance of our method, consistently exceeding the state-of-the-art methods by a large margin (e.g., 67.0% v.s. 78.9% Jaccard on Cholec80 dataset).

23 May 2022 in IEEE Transactions on Medical Imaging
Verified
Journal Article
Closed access
YJYueming Jin
Yueming Jin
YYYang Yu
Yang Yu
CCCheng Chen
Cheng Chen
ZZZixu Zhao
Zixu Zhao
PHPheng-Ann Heng
Pheng-Ann Heng
DSDanail Stoyanov
Danail Stoyanov
Automatic surgical scene segmentation is fundamental for facilitating cognitive intelligence in the modern operating theatre. Previous works rely on conventional aggregation modules (e.g., dilated convolution, convolutional LSTM), which only make use of the local context. In this paper, we propose a novel framework STswinCL that explores the complementary intra- and inter-video relations to boost segmentation performance, by progressively capturing the global context. We firstly develop a hierarchy Transformer to capture intra-video relation that includes richer spatial and temporal cues from neighbor pixels and previous frames. A joint space-time window shift scheme is proposed to efficiently aggregate these two cues into each pixel embedding. Then, we explore inter-video relation via pixel-to-pixel contrastive learning, which well structures the global embedding space. A multi-source contrast training objective is developed to group the pixel embeddings across videos with the ground-truth guidance, which is crucial for learning the global property of the whole data. We extensively validate our approach on two public surgical video benchmarks, including EndoVis18 Challenge and CaDIS dataset. Experimental results demonstrate the promising performance of our method, which consistently exceeds previous state-of-the-art approaches. Code is available at https://github.com/YuemingJin/STswinCL.

29 December 2021 in IEEE Transactions on Automation Science and Engineering
Verified
Journal Article
Closed access
Bo LuBo Lu
Bo Lu
BLBin Li
Bin Li
WCWei Chen
Wei Chen
YJYueming Jin
Yueming Jin
ZZZixu Zhao
Zixu Zhao
YLYunhui Liu
Yunhui Liu
To realize a higher-level autonomy of surgical knot tying in minimally invasive surgery (MIS), automated suture grasping, which bridges the suture stitching and looping procedures, is an important yet challenging task needs to be achieved. This paper presents a holistic framework with image-guided and automation techniques to robotize this operation even under complex environments. The whole task is initialized by suture segmentation, in which we propose a novel semi-supervised learning architecture featured with a suture-aware loss to pertinently learn its slender information using both annotated and unannotated data. With successful segmentation in stereo-camera, we develop a Sampling-based Sliding Pairing (SSP) algorithm to online optimize the suture's 3D shape. By jointly studying the robotic configuration and the suture's spatial characteristics, a target function is introduced to find the optimal grasping pose of the surgical tool with Remote Center of Motion (RCM) constraints. To compensate for inherent errors and practical uncertainties, a unified grasping strategy with a novel vision-based mechanism is introduced to autonomously accomplish this grasping task. Our framework is extensively evaluated from learning-based segmentation, 3D reconstruction, and image-guided grasping on the da Vinci Research Kit (dVRK) platform, where we achieve high performances and successful rates in perceptions and robotic manipulations. These results prove the feasibility of our approach in automating the suture grasping task, and this work fills the gap between automated surgical stitching and looping, stepping towards a higher-level of task autonomy in surgical knot tying.

10 October 2019
Verified
Book Chapter
Closed access
PDF file
ZZZixu Zhao
Zixu Zhao
HLHuangjing Lin
Huangjing Lin
HCHao Chen
Hao Chen
PHPheng-Ann Heng
Pheng-Ann Heng

14 June 2021
Verified
Book Chapter
Closed access
PDF file
XGXiaojie Gao
Xiaojie Gao
YJYueming Jin
Yueming Jin
ZZZixu Zhao
Zixu Zhao
QDQi Dou
Qi Dou
PHPheng-Ann Heng
Pheng-Ann Heng

17 February 2022 in arXiv
Verified
Preprint
Open access
PDF file
ZZZixu Zhao
Zixu Zhao
YJYueming Jin
Yueming Jin
PHPheng-Ann Heng
Pheng-Ann Heng
Surgical instrument segmentation -- in general a pixel classification task -- is fundamentally crucial for promoting cognitive intelligence in robot-assisted surgery (RAS). However, previous methods are struggling with discriminating instrument types and instances. To address the above issues, we explore a mask classification paradigm that produces per-segment predictions. We propose TraSeTR, a novel Track-to-Segment Transformer that wisely exploits tracking cues to assist surgical instrument segmentation. TraSeTR jointly reasons about the instrument type, location, and identity with instance-level predictions i.e., a set of class-bbox-mask pairs, by decoding query embeddings. Specifically, we introduce the prior query that encoded with previous temporal knowledge, to transfer tracking signals to current instances via identity matching. A contrastive query learning strategy is further applied to reshape the query feature space, which greatly alleviates the tracking difficulty caused by large temporal variations. The effectiveness of our method is demonstrated with state-of-the-art instrument type segmentation results on three public datasets, including two RAS benchmarks from EndoVis Challenges and one cataract surgery dataset CaDIS.