Eseoghene Ben-Iwhiwhu

Eseoghene Ben-Iwhiwhu

Auto-generated profile, based on publicly available information

Eseoghene Ben-Iwhiwhu

Department of Computer Science Loughborough University UK

Research Focus & Metrics

Academic Career Length

6Years (2021-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
22 March 2024 in Nature Machine Intelligence
Verified
Journal Article
Closed access
PDF file
ASAndrea Soltoggio
Andrea Soltoggio
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
VBVladimir Braverman
Vladimir Braverman
EEEric Eaton
Eric Eaton
BEBenjamin Epstein
Benjamin Epstein
SKSoheil Kolouri
Soheil Kolouri

24 September 2021 in IEEE Transactions on Neural Networks and Learning Systems
Verified
Journal Article
Closed access
PLPawel Ladosz
Pawel Ladosz
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
JDJeffery Dick
Jeffery Dick
NKNicholas Ketz
Nicholas Ketz
SKSoheil Kolouri
Soheil Kolouri
ASAndrea Soltoggio
Andrea Soltoggio
In this article, we consider a subclass of partially observable Markov decision process (POMDP) problems which we termed confounding POMDPs. In these types of POMDPs, temporal difference (TD)-based reinforcement learning (RL) algorithms struggle, as TD error cannot be easily derived from observations. We solve these types of problems using a new bio-inspired neural architecture that combines a modulated Hebbian network (MOHN) with deep Q-network (DQN), which we call modulated Hebbian plus Q-network architecture (MOHQA). The key idea is to use a Hebbian network with rarely correlated bio-inspired neural traces to bridge temporal delays between actions and rewards when confounding observations and sparse rewards result in inaccurate TD errors. In MOHQA, DQN learns low-level features and control, while the MOHN contributes to high-level decisions by associating rewards with past states and actions. Thus, the proposed architecture combines two modules with significantly different learning algorithms, a Hebbian associative network and a classical DQN pipeline, exploiting the advantages of both. Simulations on a set of POMDPs and on the Malmo environment show that the proposed algorithm improved DQN’s results and even outperformed control tests with advantage-actor critic (A2C), quantile regression DQN with long short-term memory (QRDQN + LSTM), Monte Carlo policy gradient (REINFORCE), and aggregated memory for reinforcement learning (AMRL) algorithms on most difficult POMDPs with confounding stimuli and sparse rewards.

12 April 2022 in Neural Networks
Verified
Journal Article
Open access
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
JDJeffery Dick
Jeffery Dick
NKNicholas A. Ketz
Nicholas A. Ketz
PPPraveen Pilly
Praveen Pilly
ASAndrea Soltoggio
Andrea Soltoggio
Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-art meta-RL algorithms (CAVIA and PEARL). The result demonstrates that meta-RL augmented with neuromodulation produces significantly better result and richer dynamic representations in comparison to the baselines.

1 August 2023 in arXiv
Verified
Preprint
Open access
PDF file
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
SNSaptarshi Nath
Saptarshi Nath
PPPraveen Pilly
Praveen Pilly
SKSoheil Kolouri
Soheil Kolouri
ASAndrea Soltoggio
Andrea Soltoggio
Lifelong learning aims to create AI systems that continuously and incrementally learn during a lifetime, similar to biological learning. Attempts so far have met problems, including catastrophic forgetting, interference among tasks, and the inability to exploit previous knowledge. While considerable research has focused on learning multiple supervised classification tasks that involve changes in the input distribution, lifelong reinforcement learning (LRL) must deal with variations in the state and transition distributions, and in the reward functions. Modulating masks with a fixed backbone network, recently developed for classification, are particularly suitable to deal with such a large spectrum of task variations. In this paper, we adapted modulating masks to work with deep LRL, specifically PPO and IMPALA agents. The comparison with LRL baselines in both discrete and continuous RL tasks shows superior performance. We further investigated the use of a linear combination of previously learned masks to exploit previous knowledge when learning new tasks: not only is learning faster, the algorithm solves tasks that we could not otherwise solve from scratch due to extremely sparse rewards. The results suggest that RL with modulating masks is a promising approach to lifelong learning, to the composition of knowledge to learn increasingly complex tasks, and to knowledge reuse for efficient and faster learning.

26 April 2022 in arXiv
Verified
Preprint
Open access
PDF file
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
JDJeffery Dick
Jeffery Dick
NKNicholas A. Ketz
Nicholas A. Ketz
PPPraveen Pilly
Praveen Pilly
ASAndrea Soltoggio
Andrea Soltoggio
Meta-reinforcement learning (meta-RL) algorithms enable agents to adapt quickly to tasks from few samples in dynamic environments. Such a feat is achieved through dynamic representations in an agent's policy network (obtained via reasoning about task context, model parameter updates, or both). However, obtaining rich dynamic representations for fast adaptation beyond simple benchmark problems is challenging due to the burden placed on the policy network to accommodate different policies. This paper addresses the challenge by introducing neuromodulation as a modular component to augment a standard policy network that regulates neuronal activities in order to produce efficient dynamic representations for task adaptation. The proposed extension to the policy network is evaluated across multiple discrete and continuous control environments of increasing complexity. To prove the generality and benefits of the extension in meta-RL, the neuromodulated network was applied to two state-of-the-art meta-RL algorithms (CAVIA and PEARL). The result demonstrates that meta-RL augmented with neuromodulation produces significantly better result and richer dynamic representations in comparison to the baselines.

21 January 2023 in arXiv
Verified
Preprint
Open access
PDF file
ASAndrea Soltoggio
Andrea Soltoggio
EBEseoghene Ben-Iwhiwhu
Eseoghene Ben-Iwhiwhu
CPChristos Peridis
Christos Peridis
PLPawel Ladosz
Pawel Ladosz
JDJeffery Dick
Jeffery Dick
SKSoheil Kolouri
Soheil Kolouri
This paper introduces a set of formally defined and transparent problems for reinforcement learning algorithms with the following characteristics: (1) variable degrees of observability (non-Markov observations), (2) distal and sparse rewards, (3) variable and hierarchical reward structure, (4) multiple-task generation, (5) variable problem complexity. The environment provides 1D or 2D categorical observations, and takes actions as input. The core structure of the CT-graph is a multi-branch tree graph with arbitrary branching factor, depth, and observation sets that can be varied to increase the dimensions of the problem in a controllable and measurable way. Two main categories of states, decision states and wait states, are devised to create a hierarchy of importance among observations, typical of real-world problems. A large observation set can produce a vast set of histories that impairs memory-augmented agents. Variable reward functions allow for the easy creation of multiple tasks and the ability of an agent to efficiently adapt in dynamic scenarios where tasks with controllable degrees of similarities are presented. Challenging complexity levels can be easily achieved due to the exponential growth of the graph. The problem formulation and accompanying code provide a fast, transparent, and mathematically defined set of configurable tests to compare the performance of reinforcement learning algorithms, in particular in lifelong learning settings.