Skip to main content

NavRL

Source: https://github.com/Zhefan-Xu/NavRL

Welcome to the NavRL repository! This repository provides an implementation of the NavRL framework, which aims to enable robots to safely navigate dynamic environments using reinforcement learning. While the original paper focuses on drone navigation, NavRL can be extended to any robot with a velocity-based control system.

For more details, see the related paper here:

Zhefan Xu, Xinming Han, Haoyu Shen, Hanyu Jin, and Kenji Shimada, “NavRL: Learning Safe Flight in Dynamic Environments,” IEEE Robotics and Automation Letters (RA-L), 2025. [IEEE Xplore] [Preprint] [YouTube] [BiliBili]

News​

  • **2025-04-06:** We released an easy-to-run Python script that allows users to quickly run the demo.
  • 2025-02-23: The GitHub code, video demo, and accompanying paper for our NavRL framework have been released. The authors will actively maintain and update this repository!

Table of Contents​

A quick NavRL​

We provide pre-trained models and easy-to-run Python scripts for quick demonstrations of the NavRL framework.

First, follow the steps in Deploy a virtual environment to set up a Conda environment. Once you have set up, you can run the following three demos using the following commands:

1.​

This section describes the steps for training your own RL agent using the NavRL framework in Isaac Sim. If you do not want to train the agent yourself, skip this section and jump directly to the deployment section.

Isaac Sim​

This project was developed using Isaac Sim version 2023.1.0-hotfix.1, released in November 2023. Please ensure you download and use the correct version, as using other versions may cause errors due to version incompatibilities. Also, ensure you have conda installed.

If you have already downloaded Isaac Sim 2023.1.0-hotfix.1, you can skip the following steps. Otherwise, follow the instructions below to download an older version of Isaac Sim, as the official installer does not support older versions.

To download Isaac Sim version 2023.1.0-hotfix.1:

a. First, follow the steps on this link to complete the Docker container setup.

b. Then, download Isaac Sim into your docker container:

c. Move the downloaded Isaac Sim from the docker container to your local computer:

bash docker ps # check your container ID in another terminal

# Replace <id_container> with the output from the previous command
docker cp <id_container>:isaac-sim/. /path/to/local/folder # absolute path

Isaac Sim version 2023.1.0-hotfix.1 is now installed on your local computer.

To set up the NavRL framework, clone the repository and follow these steps (this process may take several minutes):

Once the installation is complete, you should have created a virtual environment called NavRL.

Verify the installation and run the training​

Verify the installation by running the training example with 2 robots using default parameters.

If the repo is installed correctly, you should be able to see the Isaac Sim window as shown below:

isaac training window

Training Your Own RL​

The training environment settings and hyperparameters can be found in NavRL/isaac-training/training/cfg.

The following example demonstrates how to train with 1024 robots, 350 static obstacles, and 80 dynamic obstacles (requires RTX 4090). We recommend using Wandb to monitor your training and evaluation statistics.

python training/scripts/train.py headless=True env.num_envs=1024 env.num_obstacles=350 \
env_dyn.num_obstacles=80 wandb.mode=online

After enough training time, you should observe that the robot learns to avoid collisions:

isaac-training.mp4

2. Deploy a virtual​

This section provides the minimum conda environment setup required for deployment NavRL (including running on a real robot). If you have already created a conda environment in the Isaac training step NavRL, you can skip this section. To create a conda environment, run the following command:

Once the installation is complete, NavRL a conda environment named will be created. You can verify that the installation was successful by activating the environment:

III. NavRL ROS1​

This section demonstrates how to deploy NavRL on a quadcopter robot using ROS1 and Gazebo. Please ensure that your system meets the following requirements:

  • Ubuntu 20.04 LTS
  • ROS1 cognition

First, install the dependencies and ros1 copy the folder from this repository to your catkin workspace.

sudo apt-get install ros-noetic-mavros*

cp -r ros1 /path/to/catkin_ws/src
catkin_make

Then, set the environment variables for the Gazebo model.

echo 'source /path/to/ros1/uav_simulator/gazeboSetup.bash' >> ~/.bashrc

Note that the environment variables should be set in catkin_ws. For example, the correct ~./.bashrc script should source ~/catkin_ws/src/ros1/uav_simulator/gazeboSetup.bash be source path/to/NavRL/ros1/uav_simulator/gazeboSetup.bash.

Finally, start the simulation and deploy NavRL navigation.

The Gazebo window will display the environment, while the RViz window will display the data. Use RViz's 2D Nav Goal tools to set the navigation target, as shown in the video below (note: the default environment and settings may differ from the video):

ros1-deployment.mp4

To change the environment settings, look in the startup files ros1/uav_simulator/launch/start.launch. Parameters for each module are located ros1/navigation_runner/cfg/*.yaml in the configuration files.

4. NavRL ROS2​

This section demonstrates how to deploy NavRL, ROS2, and Isaac Sim using a Unitree Go2 quadruped robot. Please ensure that your system meets the following requirements:

  • Ubuntu 22.04 LTS
  • ROS2 Humility

Before you begin, install the emulator from this link.

First, ros2 copy the folders in this repository to your ros2 workspace.

cp -r ros2 /path/to/ros2_ws/src
colcon build --symlink-install

Then, start the simulation and deploy NavRL navigation.

The Isaac Sim window will display the environment, while the RViz window will display the data. Use RViz's 2D Nav Goal tools to set the navigation target. An example of navigation is shown in the following video:

go2-navigation.mp4

Note that if you want to deploy in a real robot ros2 foxy, please switch to ros2-foxy the branch.

git checkout ros2-foxy

5. Citations and​

If our work is useful for your research, please consider citing our paper.

VI.​

The authors would like to express their sincere gratitude to Professor Kenji Shimada for his strong support and all the members of the CERLAB UAV team who contributed to the development of this research.

The Isaac Sim training component of the NavRL framework is built on top of OmniDrones.