Pablo Samuel Castro

Pablo Samuel Castro

Auto-generated profile, based on publicly available information

Pablo Samuel Castro

Google DeepMind

Research Focus & Metrics

Academic Career Length

8Years (2019-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
6 February 2025 in bioRxiv
Verified
Preprint
Open access
PDF file
PCPablo Samuel Castro
Pablo Samuel Castro
NTNenad Tomasev
Nenad Tomasev
AAAnkit Anand
Ankit Anand
NSNavodita Sharma
Navodita Sharma
RMRishika Mohanta
Rishika Mohanta
KSKimberly L Stachenfeld
Kimberly L Stachenfeld
Symbolic models play a key role in cognitive science, expressing computationally precise hypotheses about how the brain implements a cognitive process. Identifying an appropriate model typically requires a great deal of effort and ingenuity on the part of a human scientist. Here, we adapt FunSearch Romera-Paredes et al. (2024), a recently developed tool that uses Large Language Models (LLMs) in an evolutionary algorithm, to automatically discover symbolic cognitive models that accurately capture human and animal behavior. We consider datasets from three species performing a classic reward-learning task that has been the focus of substantial modeling effort, and find that the discovered programs outperform state-of-the-art cognitive models for each. The discovered programs can readily be interpreted as hypotheses about human and animal cognition, instantiating interpretable symbolic learning and decision-making algorithms. Broadly, these results demonstrate the viability of using LLM-powered program synthesis to propose novel scientific hypotheses regarding mechanisms of human and animal cognition.

7 February 2025 in ACM Transactions on Economics and Computation
Verified
Journal Article
Open access
PCPablo Castro
Pablo Castro
ADAjit Desai
Ajit Desai
HDHan Du
Han Du
RGRodney Garratt
Rodney Garratt
FRFrancisco Rivadeneyra
Francisco Rivadeneyra
This article uses reinforcement learning (RL) to approximate the policy rules of banks participating in a high-value payment system (HVPS). The objective of the RL agents is to learn a policy function for the choice of amount of liquidity provided to the system at the beginning of the day and the rate at which to pay intraday payments. Individual choices have complex strategic effects precluding a closed form solution of the optimal policy, except in simple cases. We show that, in a stylized two-agent setting, RL agents learn the optimal policy that minimizes the cost of processing their individual payments—without complete knowledge of the environment. We further demonstrate that, in more complex settings, both agents learn to reduce the cost of processing their payments and effectively respond to liquidity-delay tradeoff. Our results show the potential of RL to solve liquidity management problems in HVPS and provide new tools to assist policymakers in their mandates of ensuring safety and improving the efficiency of payment systems.

21 May 2026 in bioRxiv
Verified
Preprint
Open access
PDF file
DKDaniel Kasenberg
Daniel Kasenberg
PCPablo Samuel Castro
Pablo Samuel Castro
MEMaria K. Eckstein
Maria K. Eckstein
NENoemi Elteto
Noemi Elteto
WDWill Dabney
Will Dabney
KMKevin J. Miller
Kevin J. Miller
Scientific models are widely used across the natural sciences as an interface between scientific theories and empirical data. Such models play a key role, for example, in the study of human and animal learning, where they express algorithmic hypotheses and relate them to psychology and neuroscience data. These models are traditionally handcrafted by expert researchers based on existing theory or new insights. Such handcrafted models, however, are now known to fall short of capturing the full richness of behavior, even in their narrow domains. An alternative data-driven approach has emerged, seeking to discover new insights by fitting and interpreting flexible models. However, these tools require substantial human effort to derive insight from data, and it has been unclear how to discover new ideas from data efficiently. Here, we present DataDIVER, a general approach for automatically discovering computational models from data, and demonstrate that these models surface novel mechanistic insights into human and animal learning. Our approach delivers models that take the form of short computer programs, which are optimized both to fit data well and to be simple. These programs explicitly connect with existing theoretical frameworks and are readily understandable by human scientists. They can also be used to make novel predictions, some of which we show are borne out in re-analysis of existing data. General-purpose tools for surfacing new ideas from data, especially in combination with the large datasets that are increasingly available in many fields, stand to dramatically accelerate scientific discovery.

8 November 2021 in arXiv
Verified
Preprint
Open access
PDF file
PCPablo Samuel Castro
Pablo Samuel Castro
In this paper I present a study in using the losses and gradients obtained during the training of a simple function approximator as a mechanism for creating musical dissonance and visual distortion in a solo piano performance setting. These dissonances and distortions become part of an artistic performance not just by affecting the visualizations, but also by affecting the artistic musical performance. The system is designed such that the performer can in turn affect the training process itself, thereby creating a closed feedback loop between two processes: the training of a machine learning model and the performance of an improvised piano piece.

26 June 2019 in arXiv
Verified
Preprint
Open access
PDF file
MBMarc G. Bellemare
Marc G. Bellemare
WDWill Dabney
Will Dabney
RDRobert Dadashi
Robert Dadashi
ATAdrien Ali Taiga
Adrien Ali Taiga
PCPablo Samuel Castro
Pablo Samuel Castro
CLClare Lyle
Clare Lyle
We propose a new perspective on representation learning in reinforcement learning based on geometric properties of the space of value functions. We leverage this perspective to provide formal evidence regarding the usefulness of value functions as auxiliary tasks. Our formulation considers adapting the representation to minimize the (linear) approximation of the value function of all stationary policies for a given environment. We show that this optimization reduces to making accurate predictions regarding a special class of value functions which we call adversarial value functions (AVFs). We demonstrate that using value functions as auxiliary tasks corresponds to an expected-error relaxation of our formulation, with AVFs a natural candidate, and identify a close relationship with proto-value functions (Mahadevan, 2005). We highlight characteristics of AVFs and their usefulness as auxiliary tasks in a series of experiments on the four-room domain.

3 December 2021 in arXiv
Verified
Preprint
Open access
PDF file
ZHZack Harris
Zack Harris
LCLiam Atticus Clarke
Liam Atticus Clarke
PGPietro Gagliano
Pietro Gagliano
DCDante Camarena
Dante Camarena
MSManal Siddiqui
Manal Siddiqui
PCPablo S. Castro
Pablo S. Castro
The advent of ML music models such as Google Magenta's MusicVAE now allow us to extract and replicate compositional features from otherwise complex datasets. These models allow computational composers to parameterize abstract variables such as style and mood. By leveraging these models and combining them with procedural algorithms from the last few decades, it is possible to create a dynamic song that composes music in real-time to accompany interactive experiences. Malakai is a tool that helps users of varying skill levels create, listen to, remix and share such dynamic songs. Using Malakai, a Composer can create a dynamic song that can be interacted with by a Listener