Ian Gemp

Ian Gemp

Auto-generated profile, based on publicly available information

Ian Gemp

DeepMind, London, U.K.

Research Focus & Metrics

Academic Career Length

10Years (2017-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
1 July 2017
Verified
Conference Paper
Closed access
MPMario Parente
Mario Parente
IGIan Gemp
Ian Gemp
IDIshan Durugkar
Ishan Durugkar
Spectral datasets acquired for unmixing are noisy and largely unlabeled (with unknown abundances). As a result the ability to accurately predict endmember abundances of surface samples is as important as the capacity to generate spectra from hypothetical abundances, e.g. endmembers from abundances sampled from the corner of a simplex. We construct a deep (semi-supervised) generative model to accomplish both these tasks while making use of the readily available unlabeled spectra and being able to encode environmental and instrumental nuisances. Our main technical contribution is that we train our model both forward and in reverse. The algorithm successfully identifies endmember spectra while isolating photometric effects and imaging errors as nuisances in a real dataset of intimately mixed samples.

26 June 2023 in Proceedings of the AAAI Conference on Artificial Intelligence
Verified
Journal Article
Open access
PDF file
KDKevin Du
Kevin Du
IGIan Gemp
Ian Gemp
YWYi Wu
Yi Wu
YWYingying Wu
Yingying Wu
Reinforcement learning has been used to approach well-known NP-hard combinatorial problems in graph theory. Among these, Hamiltonian cycle problems are exceptionally difficult to analyze, even when restricted to individual instances of structurally complex graphs. In this paper, we use Monte Carlo Tree Search (MCTS), the search algorithm behind many state-of-the-art reinforcement learning algorithms such as AlphaZero, to create autonomous agents that learn to play the game of Snake, a game centered on properties of Hamiltonian cycles on grid graphs. The game of Snake can be formulated as a single-player discounted Markov Decision Process (MDP), where the agent must behave optimally in a stochastic environment. Determining the optimal policy for Snake, defined as the policy that maximizes the probability of winning -- or win rate -- with higher priority and minimizes the expected number of time steps to win with lower priority, is conjectured to be NP-hard. Performance-wise, compared to prior work in the Snake game, our algorithm is the first to achieve a win rate over 0.5 (a uniform random policy achieves a win rate < 2.57 x 10^{-15}), demonstrating the versatility of AlphaZero in tackling NP-hard problems.

13 September 2023 in IEEE Transactions on Mobile Computing
Verified
Journal Article
Closed access
ZYZheng Yu
Zheng Yu
JZJunyu Zhang
Junyu Zhang
ZWZheng Wen
Zheng Wen
ATAndrea Tacchetti
Andrea Tacchetti
MWMengdi Wang
Mengdi Wang
IGIan Gemp
Ian Gemp
Complex reinforcement learning (RL) tasks often require a divide-and-conquer approach, where a large task is divided into pieces and solved by individual agents. In this paper, we study a teamwork RL setting where individual agents make decisions on disjoint subsets (blocks) of the state space and have private interests (reward functions), while the entire team aims to maximize a general long-term team utility function and may be subject to constraints. This team utility, which is not necessarily a cumulative sum of rewards, is modeled as a nonlinear function of the team's joint state-action occupancy distribution. By leveraging the inherent duality of policy optimization, we propose a min-max multi-block policy optimization framework to decompose the overall problem into individual local tasks. This enables a federated teamwork mechanism where a team lead coordinates individual agents via reward shaping, and each agent solves its local task defined only on its local state subset. We analyze the convergence of this teamwork policy optimization mechanism and establish an $O(1/T)$ convergence rate to the team's joint optimum. This mechanism allows team members to jointly find the global socially optimal policy while keeping their local privacy.