General Navigation Models

Dhruv Shah, Ajay Sridhar, Nitish Dashora, Catherine Glossop,
Kyle Stachowicz, Arjun Bhorkar, Kevin Black, Noriaki Hirose, Sergey Levine
Talk Code Data BibTex

General Navigation Models are general-purpose goal-conditioned visual navigation policies trained on diverse, cross-embodiment training data, and can control many different robots in zero-shot. They can also be efficiently fine-tuned, or adapted, to new robots and downstream tasks.

Papers



GNM: A General Navigation Model to Drive Any Robot

Dhruv Shah†, Ajay Sridhar†, Arjun Bhorkar, Noriaki Hirose, Sergey Levine
International Conference on Robotics and Automation (ICRA) 2023
London, UK
arXiv Summary Code Data

ViNT: A Foundation Model for Visual Navigation

Dhruv Shah†, Ajay Sridhar†, Nitish Dashora†,
Kyle Stachowicz, Kevin Black, Noriaki Hirose, Sergey Levine
Oral Talk at Conference on Robot Learning (CoRL) 2023
Atlanta, Georgia

Live Demo at Conference on Robot Learning (CoRL) 2023
Live Demo at Robot Learning Workshop, NeurIPS 2023
Oral Talk at Bay Area Machine Learning Symposium (BayLearn) 2023
arXiv Summary Code Data

NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration

Ajay Sridhar, Dhruv Shah, Catherine Glossop, Sergey Levine
Best Paper Award Winner at ICRA 2024
Yokohama, Japan

Best Student Paper Award Finalist at ICRA 2024
Best Paper Award in Cognitive Robotics Finalist at ICRA 2024
Oral Talk at NeurIPS Workshop on Foundation Models for Decision Making 2023
Oral Talk at CoRL Workshop on Pre-Training for Robot Learning 2023
arXiv Summary Code Data

General Navigation Models in-the-wild

GNM, ViNT, and NoMaD are available as ready-to-deploy navigation policies, and have already been used by researchers around the world on their own robots and environments.

Used GNM-family of models in your research? Send us a video of your robot in action , and we'll add it below! We love to see our models in the wild.


The website (source code) design was adapted from Nerfies.