A Guide to Open-Source VIO Code

Zhijian Qiao

中文版(知乎) All posts

This is an English translation of my Chinese article. The list reflects the projects and my impressions when I wrote it in 2022; project status and rankings may have changed.

Introduction

Learning the classic SLAM systems is essential for a graduate student in the field. Improving them, however, can be hard when you are just starting out. Ideas rarely come from nowhere. One way to start is to look up papers that cite a classic paper on Google Scholar and see how others have changed the system. But for a beginner, reading the papers is hard enough, let alone reproducing them.

I put together this survey of open-source VIO code to make it easier to build on existing work, use it in a project, or develop an idea into a paper. The categories roughly follow each paper’s main contribution; they are not intended as a strict taxonomy.

Initialization

  • VINS-Mono-QPEP (TRO 2022) formulates initialization PnP as a Quadratic Pose Estimation Problem (QPEP), addressing the possibility that the PnP algorithm in the earlier VINS-Mono initialization SfM pipeline does not find a global optimum.

Front end

Line features

  • PL-VINS: adds line features and, at the same operating speed and low power consumption, can produce higher accuracy than VINS-Mono.
  • VINS-Mono-Optimization: adds line features to VINS-Mono and tightly optimizes points and lines together. See the accompanying blog post for details.

Additional sensors

  • VINS-Fisheye: a fisheye version of VINS-Fusion for stereo cameras with a very wide field of view, accelerated with GPU and VisionWorks. It runs in real time on an NVIDIA TX2 using additional acceleration strategies, which are described in a related Zhihu post, and can also estimate depth from fisheye images. This came from my former lab.
  • VIW-Fusion: a feature-rich VINS variant with wheel odometry. In April 2022, I considered it the best option in this category.
  • GVINS (TRO 2022): tightly fuses raw GNSS measurements with visual and inertial information for real-time, drift-free state estimation. This also came from my former lab.
  • VINS-RGBD (Sensors 2019): supports localization with monocular, stereo, or RGB-D cameras and an IMU. With RGB-D, it can build dense octree maps; it also supports importing grid maps.
  • R2DIO (RA-L 2021): RGB-D-inertial SLAM.
  • IC-GVINS (RA-L 2022): a VINS system that fuses wheel odometry, IMU, GNSS, and vision.
  • VINS-RGBD-FAST (RA-L 2022): VINS-RGBD odometry for dynamic environments and resource-constrained devices such as the AGX.

Back end

  • vins_so: an early version of VINS-Fusion.
  • DynaVINS (RA-L 2022): VINS-Fusion for dynamic environments.
  • VID-Fusion (ICRA 2021): uses a tightly coupled visual-inertial-dynamics estimator to jointly estimate multirotor odometry and external forces. This is work from Prof. Fei Gao’s group.
  • VINS-Fusion with Cerebro (RAS 2021): a loop-detection method from my former lab.
  • Basalt (RA-L 2019): work from TUM using nonlinear factor recovery in the back end. It is not derived from VINS, but its improvements could be ported to VINS.
  • DM-VIO (RA-L 2022): delayed-marginalization odometry from TUM. It is not derived from VINS either, but its ideas could also be ported.
  • ROSEFusion (SIGGRAPH 2021): depth-image and IMU odometry that handles fast motion; it uses stochastic optimization to deal with large inter-frame pose changes.
  • spline_vio (ICRA 2022): continuous-time VIO based on DSO.

Filter-based methods

  • OpenVINS (ICRA): an MSCKF-based VIO library that was actively maintained when I wrote this list.
  • envio (TRO 2022): iterated extended Kalman filter VIO with photometric error and stochastic linearization.
  • EqVIO: VIO based on an equivariant filter on Lie groups, designed to improve consistency.

Multi-robot systems

  • Omni-swarm: distributed omnidirectional visual-inertial-UWB state estimation, from my former lab.
  • Kimera (IJRR 2021): MIT’s C++ library for real-time metric-semantic localization and mapping. It uses camera images and inertial data to build a semantically annotated 3D mesh of the environment.
  • vins_client_server: builds on the original VINS-Mono and adds the ability to fuse sensor data from multiple robots and send it back to a server.

Code organization and teaching

  • VINS-Course: code for a DeepBlue Academy course that builds VIO from scratch.
  • vins-mono-self-improved: reorganizes and wraps the code to improve readability.
  • VINS-simplified: mostly based on VINS-Mono, with a simplified back end implemented in Eigen.
  • My_VINS: based on VINS-Mono but does not use ROS, Ceres, or G2O. Its Eigen back end implements Levenberg–Marquardt, a sliding window, and robust kernels.

Other platforms