Skip to main content

VINS-Fusion

Sources: https://github.com/HKUST-Aerial-Robotics/VINS-Fusion https://github.com/JanekDev/VINS-Fusion-ROS2-humble-arm

ROS2 Humble version of VINS-Fusion, suitable for​

Notice​

You can now run VINS-Fusion on ROS2 Humble using ros2 launch or ros2 run.

The code is mostly based on this repository by zinuok .

Will be removed (to make the package smaller):

  • GPU enable/disable functionality has also been added: see EuRoC configuration (references here and here )
  • The GPU version has some CUDA library dependencies: OpenCV with CUDA. So if you don't mind and only need the CPU version, comment out the following compiler macros on line 14 of the 'feature_tracker.h' file:
#define GPU_MODE 1

Prerequisites​

  • system
    • Ubuntu 20.04
    • ROS2 humble
  • Dependent Libraries
    • OpenCV & cv_bridge for ROS2 Humble
    • Ceres Solver-2.1.0
    • Own-3.3.9

Construction​

  1. install_external_deps.sh Install dependencies by running the script (OpenCV, Ceres, Eigen will be installed)
  2. Build the package using colcon build
colcon build --symlink-install

Play EuRoC dataset​

To download the sample EuRoC dataset package, run get_example_data.sh the script. Then use the script to convert the dataset package to ROS2 format rosbags-convert.

./get_example_data.sh
rosbags-convert data/V1_02_medium.bag --dst /output/path

If you don't have rosbags-convert the script, you can install it using the command pip install rosbags. Then, you can ros2 bag play play the package using standard commands:

ros2 bag play /data/V1_02_medium

Original Readme file​

VINS Fusion​

Optimization-based multi-sensor state​

VINS-Fusion is an optimization-based multi-sensor state estimator that enables accurate self-localization for autonomous applications (drones, cars, and AR/VR). VINS-Fusion is an extension of VINS-Mono that supports multiple visual-inertial sensor types (monocular camera + IMU, stereo camera + IMU, or even just stereo camera). We also show an example fusing VINS with GPS. Features:

  • Supports multiple sensors (stereo camera/monocular camera + IMU/stereo camera + IMU)
  • Online spatial calibration (conversion between camera and IMU)
  • Online time calibration (time offset between camera and IMU)
  • Visual loop closure

We are the top open source stereo algorithm on the KITTI odometry benchmark (January 12, 2019).

Authors: Qin Tong, Cao Shaozu, Pan Jie, Li Peiliang, and Shen Shaojie from the Aerial Robotics Group of the Hong Kong University of Science and Technology

video:

Wiens

**Related Papers: **(**The paper and code are not exactly the same)

  • Online Temporal Calibration of Monocular Visual-Inertial Systems, Tong Qin, Shaojie Shen, IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS, 2018), Best Student Paper Award pdf
  • VINS-Mono: A Robust and Versatile Monocular Visual-Inertial State Estimator, Tong Qin, Peiliang Li, Shaojie Shen, IEEE Transactions on Robotics

If you use VINS-Fusion for academic research, please cite our related papers .

1.​

1.1 Ubuntu and​

Ubuntu 64-bit 16.04 or 18.04. ROS Kinetic or Melodic. ROS installation

1.2. Ceres​

Follow the Ceres installation.

2. Building VINS-​

Clone the repository and catkin_make:

cd ~/catkin_ws/src
    git clone https://github.com/HKUST-Aerial-Robotics/VINS-Fusion.git
    cd ../
    catkin_make
    source ~/catkin_ws/devel/setup.bash

(If this step fails, try finding another clean computer or reinstalling Ubuntu and ROS)

3. EuRoC​

Download the EuRoC MAV dataset to YOUR_DATASET_FOLDER. Using MH_01 as an example, you can run VINS-Fusion with three sensor types: monocular camera + IMU, stereo camera + IMU, and stereo camera. Open four terminals and run VINS odometry, visual loop closure (optional), rviz, and play the bag file. The green path is for VIO odometry; the red path is for odometry with visual loop closure.

3.1 Monocular Camera +​

roslaunch vins vins_rviz.launch
    rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml 
    (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml 
    rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

3.2 Stereo Camera +​

roslaunch vins vins_rviz.launch
    rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml 
    (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_imu_config.yaml 
    rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

3.3 Stereo​

roslaunch vins vins_rviz.launch
    rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_config.yaml 
    (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_stereo_config.yaml 
    rosbag play YOUR_DATASET_FOLDER/MH_01_easy.bag

4. KITTI​

4.1 KITTI Odometry (Stereo​

Download the KITTI odometry dataset to YOUR_DATASET_FOLDER. Using sequence 00 as an example, open two terminals and run vins and rviz respectively. (We evaluated odometry on the KITTI benchmark, but did not use loop closure.)

roslaunch vins vins_rviz.launch
    (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml
    rosrun vins kitti_odom_test ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/

4.2 KITTI GPS Fusion (Stereo+GPS​

Download the original KITTI dataset to YOUR_DATASET_FOLDER. Use 2011_10_03_drive_0027_synced as an example. Open three terminals and run vins, global fusion, and rviz, respectively. The green path represents VIO odometry; the blue path represents GPS odometry with global fusion.

roslaunch vins vins_rviz.launch
    rosrun vins kitti_gps_test ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/ 
    rosrun global_fusion global_fusion_node

5. VINS-Fusion In-Vehicle​

Download the car package to your dataset folder. Open four terminals and run VIO odometry, visual loop closure (optional), and rviz and play the car package files. The green path is the VIO odometry; the red path is the odometry with visual loop closure.

roslaunch vins vins_rviz.launch
    rosrun vins vins_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml 
    (optional) rosrun loop_fusion loop_fusion_node ~/catkin_ws/src/VINS-Fusion/config/vi_car/vi_car.yaml 
    rosbag play YOUR_DATASET_FOLDER/car.bag

6.​

VIO is not just a software algorithm; it relies heavily on the quality of your hardware. For beginners, we recommend using a professional device with a global shutter camera and hardware synchronization capabilities to run VIO.

6.1 Configuration​

Write a configuration file for your device. You can use the configuration files for EuRoC and KITTI as examples.

6.2 Camera​

VINS-Fusion supports multiple camera models (pinhole, MEI, equidistant). You can use the camera models to calibrate your camera. We have placed some sample data under /camera_models/calibrationdata to guide you on how to calibrate.

cd ~/catkin_ws/src/VINS-Fusion/camera_models/camera_calib_example/
rosrun camera_models Calibrations -w 12 -h 8 -s 80 -i calibrationdata --camera-model pinhole

7. Docker​

To further simplify the build process, we've added Docker to our code. The Docker environment acts like a sandbox, making our code independent of the environment. To run with Docker, first make sure you have ROS and Docker installed on your computer. Then add your account to docker the group by sudo usermod -aG docker $YOUR_USER_NAME. Restart the terminal, or if Permission denied an error occurs, log out and log back in, and enter:

cd ~/catkin_ws/src/VINS-Fusion/docker
make build

Please note that the docker build process may take a while, depending on your network and machine. After VINS-Fusion is successfully built, you can use the script to run the VINS estimator run.sh. The script run.sh accepts multiple flags and parameters. The flags -k represent KITTI, -l loop fusion, -g and global fusion. You can get usage details by using ./run.sh -h. Here are some examples of this script:

# Euroc Monocualr camera + IMU
./run.sh ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml

# Euroc Stereo cameras + IMU with loop fusion
./run.sh -l ~/catkin_ws/src/VINS-Fusion/config/euroc/euroc_mono_imu_config.yaml

# KITTI Odometry (Stereo)
./run.sh -k ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/

# KITTI Odometry (Stereo) with loop fusion
./run.sh -kl ~/catkin_ws/src/VINS-Fusion/config/kitti_odom/kitti_config00-02.yaml YOUR_DATASET_FOLDER/sequences/00/

#  KITTI GPS Fusion (Stereo + GPS)
./run.sh -kg ~/catkin_ws/src/VINS-Fusion/config/kitti_raw/kitti_10_03_config.yaml YOUR_DATASET_FOLDER/2011_10_03_drive_0027_sync/

In the case of Euroc, you need to open another terminal and run your bag file. If you need to modify the code, just ./run.sh re-run it with the appropriate extension after making the changes.

8.​

We use the ceres solver for nonlinear optimization, DBoW2 for loop closure detection, a general camera model, and GeographicLib.

9.​

The source code is released under the GPLv3 license.