Fei (Travis) Dai

Dr. Fei (Travis) Dai

orcid

Research Focus & Metrics

Academic Career Length

5Years (2022-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
10 February 2025 in Electronics
Verified
Journal Article
Open access
PDF file
FDFei (Travis) Dai
Fei (Travis) Dai
MHMd Akbar Hossain
Md Akbar Hossain
YWYi Wang
Yi Wang
Driven by rapid advancements in interconnection, packaging, integration, and computing technologies, parallel and distributed systems have significantly evolved in recent years. These systems have become essential for addressing modern computational demands, offering enhanced processing power, scalability, and resource efficiency. This paper provides a comprehensive overview of parallel and distributed systems, exploring their interrelationships, their key distinctions, and the emerging trends shaping their evolution. We analyse four parallel computing paradigms—heterogeneous computing, quantum computing, neuromorphic computing, and optical computing—and examine emerging distributed systems such as blockchain, serverless computing, and cloud-native architectures. The associated challenges are highlighted, and potential future directions are outlined. This work serves as a valuable resource for researchers and practitioners aiming to stay informed about trends in parallel and distributed computing while understanding the challenges and future developments in the field.


23 November 2022 in The Journal of Supercomputing
Verified
Journal Article
Closed access
PDF file
FDFei (Travis) Dai
Fei (Travis) Dai
YCYawen Chen
Yawen Chen
ZHZhiyi Huang
Zhiyi Huang
HZHaibo Zhang
Haibo Zhang
HZHao Zhang
Hao Zhang
CXChengpeng Xia
Chengpeng Xia

7 August 2023
Verified
Conference Paper
Open access
FDFei (Travis) Dai
Fei (Travis) Dai
YCYawen Chen
Yawen Chen
ZHZhiyi Huang
Zhiyi Huang
HZHaibo Zhang
Haibo Zhang
Communication efficiency is crucial for accelerating distributed deep neural network (DNN) training. All-reduce, a vital communication primitive, is responsible for reducing model parameters in distributed DNN training. However, most existing All-reduce algorithms, designed for traditional electrical interconnect systems, fall short due to bandwidth limitations. Optical interconnects, with superior bandwidth, low transmission delay, and less power consumption, emerge as viable alternatives. We propose Wrht (Wavelength Reused Hierarchical Tree), an efficient scheme for implementing the All-reduce operation in optical interconnect systems. Wrht leverages wavelength-division multiplexing (WDM) to minimize the communication time in distributed data-parallel DNN training. We calculate the required wavelengths, minimum communication steps, and optimal communication time, considering optical communication constraints. Simulations with real-world DNN models indicate that Wrht notably reduces communication time. On average, compared with three conventional All-reduce algorithms, Wrht achieves reductions of 65.23%, 43.81%, and 82.22% respectively in optical interconnect systems, and 61.23% and 55.51% compared with two algorithms in electrical systems. This highlights Wrht’s potential to enhance communication efficiency in DNN training using optical interconnects.

3 January 2024 in IEEE Open Journal of the Communications Society
Verified
Journal Article
Open access
FDFei (Travis) Dai
Fei (Travis) Dai
YCYawen Chen
Yawen Chen
ZHZhiyi Huang
Zhiyi Huang
HZHaibo Zhang
Haibo Zhang
In the realm of parallel and distributed computation, All-gather operation, a process where each node in a distributed system gathers data from all others, is pivotal. This operation underpins various high-performance computing (HPC) applications, notably in distributed deep learning (DL), by enabling model and hybrid parallelisms. Although optical interconnection networks promise unmatched bandwidth and reliability for data transfers between distributed nodes, most current All-gather algorithms remain optimized for electrical interconnects, leading to suboptimal performance in optical contexts. This paper proposes “OpTree”, an advanced scheme distinctly designed for All-gather operation in optical interconnect systems. OpTree constructs an optimal $m$ -ary tree that minimizes communication time by determining the optimal number of communication stages. A comprehensive comparison between OpTree’s communication steps and existing All-gather algorithms is provided. Theoretical insights reveal that OpTree substantially curtails communication steps within optical interconnects. Constraints imposed by OpTree on optical communication are also elaborated. Empirical evaluations, through rigorous simulations, establish that: 1) OpTree is effective in generating an optimal m-ary tree for minimizing communication time. 2) For a 1024-node optical ring system, OpTree cuts communication time by 72.97%, 93.15%, and 86.32% against WRHT, Ring, and Neighbor Exchange (NE) schemes, respectively, tested over different message sizes. 3) With varying node counts, the reductions stand at 42.27%, 92.74%, and 85.49% against the same counterparts. 4) As the number of wavelengths increases, communication time further diminishes.

17 December 2024 in Preprints.org
Verified
Preprint
Open access
PDF file
FDFei (Travis) Dai
Fei (Travis) Dai
MHMd Akbar Hossain
Md Akbar Hossain
YWYi Wang
Yi Wang
Driven by rapid advancements in interconnection, packaging, integration, and computing technologies, parallel and distributed systems have significantly evolved in recent years. These systems have become essential for addressing modern computational demands, offering enhanced processing power, scalability, and resource efficiency. This paper provides a comprehensive overview of parallel and distributed systems, exploring their interrelationships, key distinctions, and the emerging trends shaping their evolution. We analyze four parallel computing paradigms—heterogeneous computing, quantum computing, neuromorphic computing, and optical computing—and examine emerging distributed systems such as blockchain, serverless computing, and cloud-native architectures. The associated challenges are highlighted, and potential future directions are outlined. This work serves as a valuable resource for researchers and practitioners aiming to stay informed about trends in parallel and distributed computing while understanding the challenges and future developments in the field.