Towards Real-time Semantic RGB-D SLAM in Dynamic Environments

Published: Jul 1, 2021 by

Tete Ji

We propose a real-time semantic RGB-D SLAM system for dynamic environments that is capable of detecting both known and unknown moving objects. Most of the existing visual SLAM methods heavily rely on a static world assumption and easily fail in dynamic environments. Some recent works eliminate the influence of dynamic objects by introducing deep learning-based semantic information to SLAM systems. However, such methods suffer from high computational cost and cannot handle unknown objects.

An overview of dynamic objects detection by our proposed method. In case (a), the dynamic feature points associated with both the nearly static sitting person and the walking person are detected by the combination of semantic and geometry information. In case (b), the chair pulled by the person which is an unknown object to the semantic network is also successfully detected by our method.

Test on low dynamic sequenses.

To reduce the computational cost, we only perform semantic segmentation on key-frames to remove known dynamic objects, and maintain a static map for robust camera tracking. Furthermore, we propose an efficient geometry module to detect unknown moving objects by clustering the depth image into a few regions and identifying the dynamic regions via their reprojection errors.

Test on high dynamic sequenses.

The proposed method is evaluated on public datasets and real-world conditions. To the best of our knowledge, it is one of the first semantic RGB-D SLAM systems that run in real-time on a low-power embedded platform and provide high localization accuracy in dynamic environments.

Robustness test in real scenarios.

Publications

Towards Real-time Semantic RGB-D SLAM in Dynamic Environments.
Tete Ji, Chen Wang, Lihua Xie.
2021 International Conference on Robotics and Automation (ICRA), 2021.
```
@inproceedings{ji2021towards,
  title = {Towards Real-time Semantic RGB-D SLAM in Dynamic Environments},
  author = {Ji, Tete and Wang, Chen and Xie, Lihua},
  booktitle = {2021 International Conference on Robotics and Automation (ICRA)},
  year = {2021},
  url = {https://arxiv.org/abs/2104.01316},
  website = {https://sairlab.org/dyrgbd/}
}
```
```
Ji, Tete and Wang, Chen and Xie, Lihua, "Towards Real-time Semantic RGB-D SLAM in Dynamic Environments," 2021 International Conference on Robotics and Automation (ICRA), 2021.
```