Dongqi Han

Dongqi Han

Auto-generated profile, based on publicly available information

Dongqi Han

Microsoft Research Asia

Research Focus & Metrics

Academic Career Length

9Years (2018-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 September 2020 in Neural Networks
Verified
Journal Article
Open access
DHDongqi Han
Dongqi Han
KDKenji Doya
Kenji Doya
JTJun Tani
Jun Tani
Recurrent neural networks (RNNs) for reinforcement learning (RL) have shown distinct advantages, e.g., solving memory-dependent tasks and meta-learning. However, little effort has been spent on improving RNN architectures and on understanding the underlying neural mechanisms for performance gain. In this paper, we propose a novel, multiple-timescale, stochastic RNN for RL. Empirical results show that the network can autonomously learn to abstract sub-goals and can self-develop an action hierarchy using internal dynamics in a challenging continuous control task. Furthermore, we show that the self-developed compositionality of the network enhances faster re-learning when adapting to a new task that is a re-composition of previously learned sub-goals, than when starting from scratch. We also found that improved performance can be achieved when neural activities are subject to stochastic rather than deterministic dynamics.

25 May 2024 in Nature Communications
Verified
Journal Article
Open access
PDF file
DHDongqi Han
Dongqi Han
KDKenji Doya
Kenji Doya
DLDongsheng Li
Dongsheng Li
JTJun Tani
Jun Tani
Behaving efficiently and flexibly is crucial for biological and artificial embodied agents. Behavior is generally classified into two types: habitual (fast but inflexible), and goal-directed (flexible but slow). While these two types of behaviors are typically considered to be managed by two distinct systems in the brain, recent studies have revealed a more sophisticated interplay between them. We introduce a theoretical framework using variational Bayesian theory, incorporating a Bayesian intention variable. Habitual behavior depends on the prior distribution of intention, computed from sensory context without goal-specification. In contrast, goal-directed behavior relies on the goal-conditioned posterior distribution of intention, inferred through variational free energy minimization. Assuming that an agent behaves using a synergized intention, our simulations in vision-based sensorimotor tasks explain the key properties of their interaction as observed in experiments. Our work suggests a fresh perspective on the neural mechanisms of habits and goals, shedding light on future research in decision making.

24 November 2023 in arXiv
Verified
Preprint
Open access
PDF file
JLJie Lian
Jie Lian
XLXufang Luo
Xufang Luo
CSCaihua Shan
Caihua Shan
DHDongqi Han
Dongqi Han
VVVarut Vardhanabhuti
Varut Vardhanabhuti
DLDongsheng Li
Dongsheng Li
Precision medicine tailored to individual patients has gained significant attention in recent times. Machine learning techniques are now employed to process personalized data from various sources, including images, genetics, and assessments. These techniques have demonstrated good outcomes in many clinical prediction tasks. Notably, the approach of constructing graphs by linking similar patients and then applying graph neural networks (GNNs) stands out, because related information from analogous patients are aggregated and considered for prediction. However, selecting the appropriate edge feature to define patient similarity and construct the graph is challenging, given that each patient is depicted by high-dimensional features from diverse sources. Previous studies rely on human expertise to select the edge feature, which is neither scalable nor efficient in pinpointing crucial edge features for complex diseases. In this paper, we propose a novel algorithm named \ours, which can automatically select important features to construct multiple patient similarity graphs, and train GNNs based on these graphs as weak learners in adaptive boosting. \ours{} is evaluated on two real-world medical scenarios and shows superiors performance.

4 April 2019 in bioRxiv
Verified
Preprint
Open access
PDF file
DHDongqi Han
Dongqi Han
ESErik De Schutter
Erik De Schutter
SHSungho Hong
Sungho Hong
Feedforward networks (FFN) are ubiquitous structures in neural systems and have been studied to understand mechanisms of reliable signal and information transmission. In many FFNs, neurons in one layer have intrinsic properties that are distinct from those in their pre-/postsynaptic layers, but how this affects network-level information processing remains unexplored. Here we show that layer-to-layer heterogeneity arising from lamina-specific cellular properties facilitates signal and information transmission in FFNs. Specifically, we found that signal transformations, made by neighboring layers of neurons on an input-driven spike signal, are complementary to each other. This mechanism boosts information transfer carried by a propagating spike signal, and thereby supports reliable spike signal and information transmission in a deep FFN. Our study suggests that distinct cell types in neural circuits have complementary computational functions and facilitate information processing on the whole. Significance Statement: Neural systems have many cell types that differ in properties such as size, shape, cellular mechanisms, etc. Furthermore, neurons often propagate signals to other neurons that have properties very different from their own. We investigated what this phenomenon implies in neural information processing by using computational network models, inspired by a recent experimental study on the olfactory neural pathway of fruit flies. We found that different types of neurons can perform complementary functions in a network, which boosts information transfer on the whole and supports robust, stable signal propagation in a “deep” network with many layers. Our study demonstrates that diverse cell types with different intrinsic properties are crucial for optimal signal and information transfer in neural networks.

1 October 2018 in Review of Scientific Instruments
Verified
Journal Article
Closed access
DHDongqi Han
Dongqi Han
JXJinlin Xie
Jinlin Xie
AHAZAM Hussain
AZAM Hussain
BGBingxi Gao
Bingxi Gao
CQChengming Qu
Chengming Qu
WLWandong Liu
Wandong Liu
Electron Cyclotron Emission Imaging (ECEI) is a diagnostic system which measures 2-D electron temperature profiles with high spatial-temporal resolution. Usually only the normalized electron temperature fluctuations are utilized to investigate the magnetohydrodynamics modes due to the difficulties of ECEI calibration. In this paper, we developed a self-dependent calibration method for 24 × 16 channel high-resolution ECEI on the Experimental Advanced Superconducting Tokamak. The technique of shape matching is applied to solve for the matrix of the calibration coefficients. The calibrated area is further expanded to an occupation ratio of 88% observation area by utilizing the features of sawtooth crash. The result is self-consistent and consistent with calibrated 1D ECE measurement.

1 March 2025 in arXiv
Verified
Preprint
Open access
PDF file
HLHaofei Lu
Haofei Lu
DHDongqi Han
Dongqi Han
YSYifei Shen
Yifei Shen
DLDongsheng Li
Dongsheng Li
Diffusion models have recently shown significant potential in solving decision-making problems, particularly in generating behavior plans -- also known as diffusion planning. While numerous studies have demonstrated the impressive performance of diffusion planning, the mechanisms behind the key components of a good diffusion planner remain unclear and the design choices are highly inconsistent in existing studies. In this work, we address this issue through systematic empirical experiments on diffusion planning in an offline reinforcement learning (RL) setting, providing practical insights into the essential components of diffusion planning. We trained and evaluated over 6,000 diffusion models, identifying the critical components such as guided sampling, network architecture, action generation and planning strategy. We revealed that some design choices opposite to the common practice in previous work in diffusion planning actually lead to better performance, e.g., unconditional sampling with selection can be better than guided sampling and Transformer outperforms U-Net as denoising network. Based on these insights, we suggest a simple yet strong diffusion planning baseline that achieves state-of-the-art results on standard offline RL benchmarks.