About Me

Yaodong is a machine learning researcher with ten-year working experience in both academia and industry. Currently, he is an assistant professor at Institute for AI, Peking University. His research is about reinforcement learning and multi-agent systems. He has maintained a track record of more than sixty publications at top conferences (NeurIPS, ICML, ICLR, etc) and top journals (Artificial Intelligence, National Science Review, etc), along with the best system paper award at CoRL 2020 and the best blue-sky paper award at AAMAS 2021. Before joining Peking University, he was an assistant professor at King's College London. Before KCL, he was a principal research scientist at Huawei U.K. Before Huawei, he was a senior research manager at American International Group. He holds a Ph.D. degree from University College London, an M.Sc. degree from Imperial College London and a Bachelor degree from University of Science and Technology of China.  

杨耀东博士,北京大学人工智能研究院助理教授(博导),伦敦国王大学客座助理教授,国家级高层次留学人才回国资助计划、中国科协青年托举计划获得者。科研领域包括强化学习、博弈论和多智能体系统,重点关注基于强化学习方法的智能群体博弈及群体智能涌现。他本科毕业于中国科学技术大学,并在伦敦帝国理工大学与伦敦大学学院获得硕士及博士学位,回国前他于伦敦国王大学信息学院任助理教授。他发表AI顶会论文及专利专著 60 余篇,谷歌引用1900余次。他的工作曾获机器人学习会议CoRL’20 最佳系统论文奖、多智能体顶会AAMAS’21 最具前瞻性论文奖。因在多智能体强化学习上的工作,他曾获得世界人工智能大会(WAIC 2022)云帆奖璀璨明星、ACM SIGAI China新星奖、华为英国公司最佳技术突破奖、AAAI/ACM SIGAI 优博奖参选人 (UCL唯一)。

快速了解我的科研:

      1. 一个合作博弈的通用求解框架

      2. 一个零和博弈的通用求解框架

​本组招收对强化学习感兴趣的研究助理(带薪),请垂询

 

Recent News

11/2022

Our paper gets accepted at National Science Review [IF-23]

On the complexity of computing markov perfect equilibrium in general-sum stochastic games

We prove the complexity of computing Nash Equilibrium in Markov games are PPAD-Complete.

11/2022

Three multi-agent RL papers get accepted at AAAI 2023.

 

Mutli-agent RL:

  • Learning to Shape Rewards using a Game of Two Partners

  • Subspace-Aware Exploration for Sparse-Reward Multi-Agent Tasks

  • Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency

 

 

10/2022

Talk is given at Airs in Air.

 

Game Theoretical Multi-Agent Reinforcement Learning.

09/2022

Talk is given at Techbeat.com 2022.

A General Solution Framework to Cooperative MARL.

09/2022

Seven papers got accepted at NeurIPS 2022.

Preference-based RL:

Meta-Reward-Net: Implicitly Differentiable Reward Learning for Preference-based Reinforcement Learning

Meta-RL:

A Theoretical Understanding of Gradient Bias in Meta-Reinforcement Learning

Safe RL:

Constrained Update Projection Approach to Safe Policy Optimization

Cooperative Games:

Multi-Agent Reinforcement Learning is a Sequence Modeling Problem

Zero-sum Games:

A Unified Diversity Measure for Multiagent Reinforcement Learning

New RL environments:

Towards Human-Level Bimanual Dexterous Manipulation with Reinforcement Learning

MATE: Benchmarking Multi-Agent Reinforcement Learning in Distributed Target Coverage Control

08/2022

Tutorial on Conference on Games 2022

 

Solving two-player zero-sum games through reinforcement learning

Part I

Part II

08/2022

Two​ Invited Talks were given during the summer holidays

 

CSML China 08/22:

A continuum of solutions to cooperative MARL.

CCDM China 07/22:

Training a Population of Agents.

07/2022

One paper got accepted at IROS 2022.

Fully Decentralized Model-based Policy Optimization for Networked Systems

 

We figured out how to do model-based MARL in networked systems.

05/2022

One paper got accepted at IJCAI 2022.

1.  On the Convergence of Fictitious Play: A Decomposition Approach

     We extend the convergence guarantee for the well-known fictitious play method.

04/2022

We open source two reinforcement learning projects:

1.  TorchOpt

     We develop an optimisation tool in Pytorch where meta-gradients can be                     computed easily.

     With TorchOpt, you can implement Meta-RL algorithms easily, try our code!

2.  BiDexHands

     We develop a RL/MARL environment for bimanual dexterous hands                           manipulations.

     BiDexhands are super fast, you can reach 40,000 FPS by only one GPU.

01/2022

Two papers got accepted at ICLR 2022.

 

1.  Multiagent-Agent TRPO Methods

     We develop how to conduct trust-region updates in MARL settings.

     This is the SOTA algorithm in the cooperative MARL space, try our code!

     [English Blog]      [Chinese Blog]     [Code]

2.  LIGS: Learnable Intrinsic-Reward Generation Selection for Multi-          Agent Learning

     The paper addresses coordination improvement in the MARL setting by          learning intrinsic rewards that motivate the exploration and coordination.

01/2022

北大AI院多智能体中心面向全球招收寒研学生,见课题如下。

Link

01/2022

Invited talk at DAI 2021 on the topic of Training A Population of Reinforcement Learning Agents.

Slides 

09/2021

Three papers get accepted at NeurIPS 2021:

1. https://lnkd.in/dEGYvaKq     

          (see Blog here) 

We analysed the variance of gradient norm for multi-agent reinforcement learning and developed a minimal-variance policy gradient estimator.


2. https://lnkd.in/dqcvH4Ns

We developed a rigorous way to generate diverse policies in population-based training and demonstrated impressive results on Google football. 


3. https://lnkd.in/db__AKEY

We show it is entirely possible to make AI learn to learn how to solve zero-sum games without even telling it what is a Nash equilibrium.

08/2021

Invited talk at RLChina on the tutorial of Multi-Agent Learning.

Slides  Video (in Chinese)

07/2021

Invited talk by 机器之心 on my recent work on how to deal with non-transitivity in two-player zero-sum games.

Slides  Video (in Chinese)

06/2021

We opensource MALib: A bespoke high-performance framework for population-based multi-agent reinforcement learning.

 

Paper  Github

05/2021

Two papers get accepted in ICML 2021.

Modelling Behavioural Diversity for Learning in Open-Ended Games. This paper studies how to measure and promote behavioural diversity in solving games in a mathematically rigorous way. It is awarded a long talk (top 3%) at ICML 2021.

Learning in Nonzero-Sum Stochastic Games with Potentials. This paper studies a generalised class of fully cooperative games, named stochastic potential games, and propose a MARL solution to find the Nash in such games. 

03/2021

Check out my recent talk on the topic of:

A general framework for solving two-player zero-sum games. 

02/2021

Update: Our paper wins the Best Paper Award at the Blue Sky Idea track!!!

One paper gets accepted in AAMAS 2021.

Diverse Auto-Curriculum is Critical for Successful Real-World Multiagent Learning Systems. I express some of my recent thoughts on why behavioural diversity in the policy space is an important factor for MARL techniques to be applied in real-world problems, outside purely video games. 

11/2020

Check out my latest work on:

An Overview of Multi-Agent Reinforcement Learning from Game Theoretical Perspective.  I hope this work could offer a nice summary of game theory basics for MARL researches in addition to the deep RL hype :)

10/2020

Update: SMARTS won the BEST paper award in CoRL 2020!

We release SMARTS: a multi-agent reinforcement learning enabled autonomous driving platform.

Multi-agent interaction is a fundamental aspect of autonomous driving in the real world. Today we are excited to introduce a dedicated platform: SMARTS, that supports Scalable Multi-Agent Reinforcement Learning Training for autonomous driving. With SMARTS, ML researchers can now evaluate their new algorithms in the self-driving scenarios, in addition to traditional video games. In turn, SMARTS can enrich the social vehicle behaviours and create increasingly more realistic and diverse interactions, powered by RL techniques, for autonomous driving researchers. Check our code on Github, and our paper at Conference on Robotic Learning 2020

10/2020

One paper gets accepted at NIPS 2020 !

Replica-exchange Nos\'e-Hoover dynamics for Bayesian learning on large datasets. We introduce a new HMC sampler for large-scale Bayesian deep learning that suits multi-mode sampling and the noises from mini-batches can be absorbed by a special design of Nose-Hoover dynamics.

09/2020

One paper gets accepted at CIKM 2020 !

Learning to infer user hidden states for online sequential advertising.

08/2020

A lecture was given at RL China Summer School.

Advances of Multi-agent Learning in Gaming AI.

06/2020

A talk was given at ISTBI, Fudan University.

Many-agent Reinforcement Learning.

06/2020

One paper gets accepted at ICML 2020

Multi-agent Determinantal Q-learning. We introduce a new function approximator called Q-determinant point process for multi-agent reinforcement learning problems. It can help learn the Q-function factorisation with no needs for a priori structural constraints such as QMIX, VDN, etc.

05/2020

One paper gets accepted at IJCAI 2020

Modelling Bounded Rationality in Multi-Agent Interactions by Generalized Recursive Reasoning. We use probabilisitic graphical model to describe the recursive reasoning process of "I believe you believe I believe..." in the multi-agent system.  

02/2020

One paper gets accepted at AAMAS 2020

Alpha^Alpla-Rank: Practically Scaling Alpha-Rank through Stochastic Optimisation. Alpha-Rank is a replacement for Nash equilibrium for general-sum N-player game, importantly, its solution is P-complete. In this paper, we further enhance its tractability by several orders of magnitude by stochastic optimisation formulation.