Awesome In-Context Reinforcement Learning

This is a collection of research papers for In-Context Reinforcement Learning (ICRL). The repository shall be regularly updated to track the frontiers.

Curated by dunnolab.

---

Please, feel free to PR new papers and resources you believe are relevant and awesome.

format:
- [title](paper link)
  - author1, author2, and author3...

Papers

• In-context Reinforcement Learning with Algorithm Distillation
  • Michael Laskin, Luyu Wang, Junhyuk Oh, Emilio Parisotto, Stephen Spencer, Richie Steigerwald, DJ Strouse, Steven Hansen, Angelos Filos, Ethan Brooks, Maxime Gazeau, Himanshu Sahni, Satinder Singh, Volodymyr Mnih
• Supervised Pretraining Can Learn In-Context Reinforcement Learning
  • Jonathan N. Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill
• Emergence of In-Context Reinforcement Learning from Noise Distillation
  • Ilya Zisman, Vladislav Kurenkov, Alexander Nikulin, Viacheslav Sinii, Sergey Kolesnikov
• In-Context Reinforcement Learning for Variable Action Spaces
  • Viacheslav Sinii, Alexander Nikulin, Vladislav Kurenkov, Ilya Zisman, Sergey Kolesnikov
• In-Context Reinforcement Learning Without Optimal Action Labels
  • Juncheng Dong, Moyang Guo, Ethan X Fang, Zhuoran Yang, Vahid Tarokh
• XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
  • Alexander Nikulin, Ilya Zisman, Alexey Zemtsov, Viacheslav Sinii, Vladislav Kurenkov, Sergey Kolesnikov
• ReLIC: A Recipe for 64k Steps of In-Context Reinforcement Learning for Embodied AI
  • Ahmad Elawady, Gunjan Chhablani, Ram Ramrakhya, Karmesh Yadav, Dhruv Batra, Zsolt Kira, Andrew Szot
• Sparse Autoencoders Reveal Temporal Difference Learning in Large Language Models
  • Can Demircan, Tankred Saanum, Akshay K. Jagadish, Marcel Binz, Eric Schulz
• AMAGO: Scalable In-Context Reinforcement Learning for Adaptive Agents
  • Jake Grigsby, Linxi Fan, Yuke Zhu
• Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining
  • Licong Lin, Yu Bai, Song Mei
• Structured State Space Models for In-Context Reinforcement Learning
  • Chris Lu, Yannick Schroecker, Albert Gu, Emilio Parisotto, Jakob Foerster, Satinder Singh, Feryal Behbahani
• LLMs Are In-Context Reinforcement Learners
  • Giovanni Monea, Antoine Bosselut, Kianté Brantley, Yoav Artzi
• EVOLvE: Evaluating and Optimizing LLMs For Exploration
  • Allen Nie, Yi Su, Bo Chang, Jonathan N. Lee, Ed H. Chi, Quoc V. Le, Minmin Chen
• In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought
  • Sili Huang, Jifeng Hu, Hechang Chen, Lichao Sun, Bo Yang
• Retrieval-Augmented Decision Transformer: External Memory for In-context RL
  • Thomas Schmied, Fabian Paischer, Vihang Patil, Markus Hofmarcher, Razvan Pascanu, Sepp Hochreiter
• Human-Timescale Adaptation in an Open-Ended Task Space
  • Adaptive Agent Team, Jakob Bauer, Kate Baumli, Satinder Baveja, Feryal Behbahani, Avishkar Bhoopchand, Nathalie Bradley-Schmieg, Michael Chang, Natalie Clay, Adrian Collister, Vibhavari Dasagi, Lucy Gonzalez, Karol Gregor, Edward Hughes, Sheleem Kashem, Maria Loks-Thompson, Hannah Openshaw, Jack Parker-Holder, Shreya Pathak, Nicolas Perez-Nieves, Nemanja Rakicevic, Tim Rocktäschel, Yannick Schroecker, Jakub Sygnowski, Karl Tuyls, Sarah York, Alexander Zacherl, Lei Zhang
• Generalization to New Sequential Decision Making Tasks with In-Context Learning
  • Sharath Chandra Raparthy, Eric Hambro, Robert Kirk, Mikael Henaff, Roberta Raileanu
• Large Language Models can Implement Policy Iteration
  • Ethan Brooks, Logan Walls, Richard L. Lewis, Satinder Singh
• Learning How to Infer Partial MDPs for In-Context Adaptation and Exploration
  • Chentian Jiang, Nan Rosemary Ke, Hado van Hasselt
• Towards General-Purpose In-Context Learning Agents
  • Louis Kirsch, James Harrison, C. Daniel Freeman, Jascha Sohl-Dickstein, Jürgen Schmidhuber
• Large Language Models as General Pattern Machines
  • Suvir Mirchandani, Fei Xia, Pete Florence, Brian Ichter, Danny Driess, Montserrat Gonzalez Arenas, Kanishka Rao, Dorsa Sadigh, Andy Zeng
• Cross-Episodic Curriculum for Transformer Agents
  • Lucy Xiaoyang Shi, Yunfan Jiang, Jake Grigsby, Linxi "Jim" Fan, Yuke Zhu
• First-Explore, then Exploit: Meta-Learning Intelligent Exploration
  • Ben Norman, Jeff Clune
• Meta-Reinforcement Learning Robust to Distributional Shift Via Performing Lifelong In-Context Learning
  • Tengye Xu, Zihao Li, Qinyuan Ren
• Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learning
  • Jonathan Cook, Chris Lu, Edward Hughes, Joel Z. Leibo, Jakob Foerster
• Large Language Models As Evolution Strategies
  • Robert Tjarko Lange, Yingtao Tian, Yujin Tang
• Can Large Language Models Explore In-Context?
  • Akshay Krishnamurthy, Keegan Harris, Dylan J. Foster, Cyril Zhang, Aleksandrs Slivkins
• SAD: State-Action Distillation for In-Context Reinforcement Learning under Random Policies
  • Weiqin Chen, Santiago Paternain
• Pretraining Decision Transformers with Reward Prediction for In-Context Multi-task Structured Bandit Learning
  • Subhojyoti Mukherjee, Josiah P. Hanna, Qiaomin Xie, Robert Nowak
• Transformers Learn Temporal Difference Methods for In-Context Reinforcement Learning
  • Jiuqi Wang, Ethan Blaser, Hadi Daneshmand, Shangtong Zhang