Joseph Suarez

Joseph Suarez

Auto-generated profile, based on publicly available information

Joseph Suarez

Massachusetts Institute of Technology

Research Focus & Metrics

Academic Career Length

4Years (2023-Present)

(Calculated once Scilit publication data is synced: current year − year of first publication + 1)

Representative Publications

View All Publications
7 November 2023 in arXiv
Verified
Preprint
Open access
PDF file
JSJoseph Suárez
Joseph Suárez
PIPhillip Isola
Phillip Isola
KCKyoung Whan Choe
Kyoung Whan Choe
DBDavid Bloomin
David Bloomin
HLHao Xiang Li
Hao Xiang Li
XZXiaolong Zhu
Xiaolong Zhu
Neural MMO 2.0 is a massively multi-agent environment for reinforcement learning research. The key feature of this new version is a flexible task system that allows users to define a broad range of objectives and reward signals. We challenge researchers to train agents capable of generalizing to tasks, maps, and opponents never seen during training. Neural MMO features procedurally generated maps with 128 agents in the standard setting and support for up to. Version 2.0 is a complete rewrite of its predecessor with three-fold improved performance and compatibility with CleanRL. We release the platform as free and open-source software with comprehensive documentation available at neuralmmo.github.io and an active community Discord. To spark initial research on this new platform, we are concurrently running a competition at NeurIPS 2023.

27 October 2023 in arXiv
Verified
Preprint
Open access
PDF file
RSRyan Sullivan
Ryan Sullivan
AKAkarsh Kumar
Akarsh Kumar
SHShengyi Huang
Shengyi Huang
JDJohn P. Dickerson
John P. Dickerson
JSJoseph Suarez
Joseph Suarez
Most reinforcement learning methods rely heavily on dense, well-normalized environment rewards. DreamerV3 recently introduced a model-based method with a number of tricks that mitigate these limitations, achieving state-of-the-art on a wide range of benchmarks with a single set of hyperparameters. This result sparked discussion about the generality of the tricks, since they appear to be applicable to other reinforcement learning algorithms. Our work applies DreamerV3's tricks to PPO and is the first such empirical study outside of the original work. Surprisingly, we find that the tricks presented do not transfer as general improvements to PPO. We use a high quality PPO reference implementation and present extensive ablation studies totaling over 10,000 A100 hours on the Arcade Learning Environment and the DeepMind Control Suite. Though our experiments demonstrate that these tricks do not generally outperform PPO, we identify cases where they succeed and offer insight into the relationship between the implementation tricks. In particular, PPO with these tricks performs comparably to PPO on Atari games with reward clipping and significantly outperforms PPO without reward clipping.

7 November 2023 in arXiv
Verified
Preprint
Open access
PDF file
ELEnhong Liu
Enhong Liu
JSJoseph Suarez
Joseph Suarez
CYChenhui You
Chenhui You
BWBo Wu
Bo Wu
BCBingcheng Chen
Bingcheng Chen
PIPhillip Isola
Phillip Isola
In this paper, we present the results of the NeurIPS-2022 Neural MMO Challenge, which attracted 500 participants and received over 1,600 submissions. Like the previous IJCAI-2022 Neural MMO Challenge, it involved agents from 16 populations surviving in procedurally generated worlds by collecting resources and defeating opponents. This year's competition runs on the latest v1.6 Neural MMO, which introduces new equipment, combat, trading, and a better scoring system. These elements combine to pose additional robustness and generalization challenges not present in previous competitions. This paper summarizes the design and results of the challenge, explores the potential of this environment as a benchmark for learning methods, and presents some practical reinforcement learning training approaches for complex tasks with sparse rewards. Additionally, we have open-sourced our baselines, including environment wrappers, benchmarks, and visualization tools for future research.

30 August 2023 in arXiv
Verified
Preprint
Open access
PDF file
YCYangkun Chen
Yangkun Chen
JSJoseph Suarez
Joseph Suarez
JZJunjie Zhang
Junjie Zhang
CYChenghui Yu
Chenghui Yu
BWBo Wu
Bo Wu
PIPhillip Isola
Phillip Isola
We present the results of the second Neural MMO challenge, hosted at IJCAI 2022, which received 1600+ submissions. This competition targets robustness and generalization in multi-agent systems: participants train teams of agents to complete a multi-task objective against opponents not seen during training. The competition combines relatively complex environment design with large numbers of agents in the environment. The top submissions demonstrate strong success on this task using mostly standard reinforcement learning (RL) methods combined with domain-specific engineering. We summarize the competition design and results and suggest that, as an academic community, competitions may be a powerful approach to solving hard problems and establishing a solid benchmark for algorithms. We will open-source our benchmark including the environment wrapper, baselines, a visualization tool, and selected policies for further research.

18 August 2025 in arXiv
Verified
Preprint
Open access
PDF file
JSJoseph Suárez
Joseph Suárez
KCKyoung Whan Choe
Kyoung Whan Choe
DBDavid Bloomin
David Bloomin
JGJianming Gao
Jianming Gao
YLYunkun Li
Yunkun Li
PIPhillip Isola
Phillip Isola
We present the results of the NeurIPS 2023 Neural MMO Competition, which attracted over 200 participants and submissions. Participants trained goal-conditional policies that generalize to tasks, maps, and opponents never seen during training. The top solution achieved a score 4x higher than our baseline within 8 hours of training on a single 4090 GPU. We open-source everything relating to Neural MMO and the competition under the MIT license, including the policy weights and training code for our baseline and for the top submissions.