Kefei Liu

Kefei Liu

Auto-generated profile, based on publicly available information

Kefei Liu

China Mobile Research Institute, Beijing, China

Research Focus & Metrics

Academic Career Length

3Years (2024-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
4 August 2024
Verified
Conference Paper
Open access
KLKefei Liu
Kefei Liu
ZJZhuo Jiang
Zhuo Jiang
JZJiao Zhang
Jiao Zhang
SGShixian Guo
Shixian Guo
XZXuan Zhang
Xuan Zhang
THTao Huang
Tao Huang
RoCE services are sensitive to network failures and performance bottlenecks, which become more common as the RoCE network scales. In addition, some non-network problems behave like network problems and can waste troubleshooting time. However, existing mechanisms cannot quickly detect and locate network problems or determine whether the service problem is network-related. In this paper, we propose R-Pingmesh, the first service-aware RoCE network monitoring and diagnostic system based on end-to-end probing. R-Pingmesh can accurately measure network RTT and end-host processing delay based on commodity RDMA NICs (RNICs), distinguish between RNIC and in-network packet drops, and judge whether a problem is network-related and assess its impact on services. We have deployed R-Pingmesh on tens of thousands of RNICs for over 6 months. One-month evaluation results show that 85% of the problems located by R-Pingmesh are accurate, where all 157 switch network problems are accurate. R-Pingmesh efficiently detects and locates 14 types of problems during deployment, and we share our experience in dealing with them.

27 August 2025
Verified
Conference Paper
Open access
SGShixian Guo
Shixian Guo
KLKefei Liu
Kefei Liu
YLYulin Lai
Yulin Lai
YBYangyang Bai
Yangyang Bai
ZZZiwei Zhao
Ziwei Zhao
THTao Huang
Tao Huang

3 August 2024
Verified
Conference Paper
Open access
KLKefei Liu
Kefei Liu
JZJiao Zhang
Jiao Zhang
ZJZhuo Jiang
Zhuo Jiang
XZXuan Zhang
Xuan Zhang
SGShixian Guo
Shixian Guo
THTao Huang
Tao Huang
RoCE services are sensitive to failures and bottlenecks, which become more common as the RoCE network scales. To effectively detect and locate these problems independent of service traffic, RoCE networks require a monitoring and diagnostic system based on active probing. However, existing active probing schemes typically rely on a controller to design the probing plan for each server, which is difficult to deploy and has high synchronization overhead in multi-tenant clusters. Fortunately, rail-optimized clusters have become more common in recent years to improve network performance. In these clusters, the controller is unnecessary. In this paper, we propose Hostmesh, the first network monitoring and diagnostic system for rail-optimized RoCE clusters based solely on full-mesh probing between RDMA NICs on the same host. Hostmesh uses the feature of rail-optimized networks and does not rely on a controller to generate pinglists. We deployed Hostmesh for over three months on a multi-tenant rail-optimized RoCE cluster with hundreds of servers. During the deployment, Hostmesh effectively detected and located 8 types of problems caused by hardware failures, misconfigurations, network congestion, and intra-host bottlenecks. And we share our experience in dealing with them.

16 July 2024 in IEEE/ACM Transactions on Networking
Verified
Journal Article
Closed access
KLKefei Liu
Kefei Liu
JZJiao Zhang
Jiao Zhang
ZJZhuo Jiang
Zhuo Jiang
HWHaoran Wei
Haoran Wei
XZXiaolong Zhong
Xiaolong Zhong
THTao Huang
Tao Huang
In RDMA (Remote Direct Memory Access) networks, end-host networks, including intra-host networks and RNICs (RDMA NIC), were considered robust and have received little attention. However, as the RNIC line rate rapidly increases to multi-hundred gigabits, the intra-host network becomes a potential performance bottleneck for network applications. Intra-host network bottlenecks can result in degraded intra-host bandwidth and increased intra-host latency. In addition, RNIC network problems can result in connection failures and packet drops. Host network problems can severely degrade network performance. However, when host network problems occur, they can hardly be noticed due to the lack of a monitoring system. Furthermore, existing diagnostic mechanisms cannot efficiently diagnose host network problems. In this paper, we analyze the symptom of host network problems based on our long-term troubleshooting experience and propose Hostping, the first monitoring and diagnostic system dedicated to host networks. The core idea of Hostping is to conduct 1) loopback tests between RNICs and endpoints within the host to measure intra-host latency and bandwidth, and 2) mutual probing between RNICs on a host to measure RNIC connectivity. We have deployed Hostping on thousands of servers in our distributed machine learning system. Not only can Hostping detect and diagnose host network problems we already knew in minutes, but it also reveals eight problems we did not notice before.

7 January 2025 in IEEE Transactions on Networking
Verified
Journal Article
Closed access
ZWZirui Wan
Zirui Wan
JZJiao Zhang
Jiao Zhang
YWYuxiang Wang
Yuxiang Wang
KLKefei Liu
Kefei Liu
HPHaoyu Pan
Haoyu Pan
THTao Huang
Tao Huang
RDMA has been widely deployed in production datacenters. The conventional wisdom believes that the intra-host network delivers stable and high performance. However, intra-host resources witness a relative stagnation in technology trends compared to the evolving RDMA NIC (RNIC). Thus, the RNIC traffic may not get sufficient intra-host resources when it contends with CPU-to-memory traffic. A line of recent works from large-scale production datacenter operators demonstrates the emergence of intra-host congestion and associated performance collapse, which forces us to revisit the practice of intra-host congestion control. However, the ability to efficiently control RDMA intra-host networks is far less mature than inter-host networks, which brings challenges in congestion monitoring, intra-host resource allocation and RNIC traffic adjustment. In this paper, we propose RDMA intra-Host Congestion Control (RHCC), which combines CPU-to-memory traffic congestion avoidance with sub-RTT granularity and proactive RNIC traffic adjustment. RHCC ensures fast congestion avoidance and can work with different inter-host congestion control methods. We implement RHCC on commodity servers and RNICs and conduct experiments to evaluate the performance. The results show that RHCC can increase/decrease the network throughput/latency by up to 2 $\times$ and 1.4 $\times$ , respectively.

3 August 2024
Verified
Conference Paper
Open access
ZWZirui Wan
Zirui Wan
JZJiao Zhang
Jiao Zhang
YWYuxiang Wang
Yuxiang Wang
KLKefei Liu
Kefei Liu
HPHaoyu Pan
Haoyu Pan
THTao Huang
Tao Huang
RDMA has been widely deployed in production datacenters. The conventional wisdom believes that the intra-host network delivers stable and high performance. However, intra-host resources witness a relative stagnation in technology trends compared to the evolving RDMA NIC (RNIC). Thus, the RNIC traffic may not get sufficient intra-host resources when it contends with intra-host traffic. A line of recent works from large-scale production datacenter operators demonstrates the emergence of intra-host congestion and associated performance collapse, which forces us to rethink the practice of intra-host congestion control. However, the ability to efficiently control RDMA intra-host networks is far less mature than inter-host networks, which brings challenges in congestion monitoring, intra-host resource allocation and RNIC traffic adjustment. In this paper, we propose RDMA intra-Host Congestion Control (RHCC), which combines sub-RTT granularity intra-host traffic congestion avoidance and proactive RNIC traffic adjustment. We implement RHCC on commodity servers and RNICs and conduct experiments to evaluate the performance. The results show that RHCC can increase/decrease the network throughput/latency by up to 2 × and 1.4 ×, respectively.