Mahammad Humayoo

Dr. Mahammad Humayoo

Institute of Computing Technology, CAS
orcid

Research Focus & Metrics

Academic Career Length

13Years (2014-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
12 October 2019 in Applied Sciences
Verified
Journal Article
Open access
PDF file
MHMahammad Humayoo
Mahammad Humayoo
XCXueqi Cheng
Xueqi Cheng
Regularization is a popular technique in machine learning for model estimation and for avoiding overfitting. Prior studies have found that modern ordered regularization can be more effective in handling highly correlated, high-dimensional data than traditional regularization. The reason stems from the fact that the ordered regularization can reject irrelevant variables and yield an accurate estimation of the parameters. How to scale up the ordered regularization problems when facing large-scale training data remains an unanswered question. This paper explores the problem of parameter estimation with the ordered ℓ 2 -regularization via Alternating Direction Method of Multipliers (ADMM), called ADMM-O ℓ 2 . The advantages of ADMM-O ℓ 2 include (i) scaling up the ordered ℓ 2 to a large-scale dataset, (ii) predicting parameters correctly by excluding irrelevant variables automatically, and (iii) having a fast convergence rate. Experimental results on both synthetic data and real data indicate that ADMM-O ℓ 2 can perform better than or comparable to several state-of-the-art baselines.

24 April 2025 in Scientific Reports
Verified
Journal Article
Open access
PDF file
MHMahammad Humayoo
Mahammad Humayoo
GZGengzhong Zheng
Gengzhong Zheng
XDXiaoqing Dong
Xiaoqing Dong
LMLiming Miao
Liming Miao
SQShuwei Qiu
Shuwei Qiu
XCXueqi Cheng
Xueqi Cheng
Off-policy learning exhibits greater instability when compared to on-policy learning in reinforcement learning (RL). The difference in probability distribution between the target policy (\(\pi\)) and the behavior policy (b) is a major cause of instability. High variance also originates from distributional mismatch. The variation between the target policy’s distribution and the behavior policy’s distribution can be reduced using importance sampling (IS). However, importance sampling has high variance, which is exacerbated in sequential scenarios. We propose a smooth form of importance sampling, specifically relative importance sampling (RIS), which mitigates variance and stabilizes learning. To control variance, we alter the value of the smoothness parameter \(\beta \in [0, 1]\) in RIS. We develop the first model-free relative importance sampling off-policy actor-critic (RIS-off-PAC) algorithms in RL using this strategy. Our method uses a network to generate the target policy (actor) and evaluate the current policy (\(\pi\)) using a value function (critic) based on behavior policy samples. Our algorithms are trained using behavior policy action values in the reward function, not target policy ones. Both the actor and critic are trained using deep neural networks. Our methods performed better than or equal to several state-of-the-art RL benchmarks on OpenAI Gym challenges and synthetic datasets.

20 November 2025 in Algorithms
Verified
Journal Article
Open access
PDF file
MHMahammad Humayoo
Mahammad Humayoo
GZGengzhong Zheng
Gengzhong Zheng
XDXiaoqing Dong
Xiaoqing Dong
WHWei Huang
Wei Huang
MLMiao Liming
Miao Liming
XCXueqi Cheng
Xueqi Cheng
In numerous episodic reinforcement learning (RL) environments, SARSA-based methodologies are employed to enhance policies aimed at maximizing returns over long horizons. Traditional SARSA algorithms face challenges in achieving an optimal balance between bias and variation, primarily due to their dependence on a single, constant discount factor (η). This study enhances the temporal difference decomposition method, TD(Δ), by applying it to the SARSA algorithm, wherein the action-value function is segmented into several components based on the differences between action-value functions linked to specific discount factors. Each component, referred to as a delta estimator (D), is linked to a specific discount factor and learned independently. This modified technique is referred to as SARSA(Δ). SARSA is a widely used on-policy RL method that enhances action-value functions via temporal difference updates. This decomposition, namely SARSA(Δ), facilitates learning across a range of time scales. This analysis makes learning more effective and guarantees consistency, especially in situations where long-horizon improvement is needed. The results of this research show that the proposed technique works to lower bias in SARSA’s updates and speed up convergence in both deterministic and stochastic settings, even in dense-reward Atari environments. Experimental results from a variety of benchmark settings show that the proposed SARSA(Δ) outperforms existing TD learning techniques in both tabular and deep RL environments.

1 January 2014 in Lecture Notes in Computer Science
Verified
Book Chapter
Closed access
PDF file
MHMahammad Humayoo
Mahammad Humayoo
YZYanlong Zhai
Yanlong Zhai
YHYan He
Yan He
BXBingqing Xu
Bingqing Xu
CWChen Wang
Chen Wang

22 December 2022 in arXiv
Verified
Preprint
Open access
PDF file
MHMahammad Humayoo
Mahammad Humayoo
XCXueqi Cheng
Xueqi Cheng
Off-policy learning is more unstable compared to on-policy learning in reinforcement learning (RL). One reason for the instability of off-policy learning is a discrepancy between the target ($\pi$) and behavior (b) policy distributions. The discrepancy between $\pi$ and b distributions can be alleviated by employing a smooth variant of the importance sampling (IS), such as the relative importance sampling (RIS). RIS has parameter $\beta\in[0, 1]$ which controls smoothness. To cope with instability, we present the first relative importance sampling-off-policy actor-critic (RIS-Off-PAC) model-free algorithms in RL. In our method, the network yields a target policy (the actor), a value function (the critic) assessing the current policy ($\pi$) using samples drawn from behavior policy. We use action value generated from the behavior policy in reward function to train our algorithm rather than from the target policy. We also use deep neural networks to train both actor and critic. We evaluated our algorithm on a number of Open AI Gym benchmark problems and demonstrate better or comparable performance to several state-of-the-art RL baselines.

1 August 2018
Verified
Conference Paper
Closed access
MHMahammad Humayoo
Mahammad Humayoo
XCXueqi Cheng
Xueqi Cheng
Automatic selection of true explanatory variables and controlling fraction of false discovery rate (FDR) in the linear model has received considerable attention in machine learning. The ordered regularization is an important component of the linear model and plays a key role in solving such kind of problems. Although there have been some models proposed for determining relevant features in either low-dimensional or high-dimensional space. However, there exists no single sorted model that can work in both low-high dimensions cases. This paper introduces a model called mSLOPE (model-free SLOPE) which is a mixed methodology based on model-free (MF) knockoffs and sorted Lone penalized estimation (SLOPE). mSLOPE uses original design matrix augmented with MF knockoffs matrix. The original feature matrix and MF matrix have the same covariance structure. Advantages of mSLOPE include, (i) it identifies true regressors in any dimension, (ii) it is an adaptive and computationally tractable and (iii) it gains power relative to its competitors through an exact control of FDR in any dimensions. Experimental results on both synthetic and real data show that mSLOPE gains superior power and accurate FDR control than state-of-the-art baselines.