Simultaneous localization and mapping (SLAM) is a fundamental task for numerous applications such as autonomous navigation and exploration. Despite many SLAM datasets have been released, current SLAM solutions still struggle to have sustained and resilient performance. One major issue is the absence of high-quality datasets including diverse all-weather conditions and a reliable metric for assessing robustness. This limitation significantly restricts the scalability and generalizability of SLAM technologies, impacting their development, validation, and deployment.
To bridge this gap and push SLAM towards all-weather environments, we present an extremely challenging dataset, SubT-MRS, including scenarios featuring various sensor degradation, aggressive locomotions, and extreme-weather conditions. The SubT-MRS dataset comprises 3 years of data from the DARPA Subterranean (SubT) Challenge (2019-2021) and extends with an additional 2 years of diverse environments (2022-2023), containing mixed indoor and outdoor settings, including long corridors, off-road scenario, tunnels, caves, deserts, forests, and bushlands.
Cumulatively, this forms a 5-year dataset encompassing over 2000 hours and 300 kilometers of terrain subjected to multimodal sensors including LiDAR, fisheye cameras, depth cameras, thermal cameras, and IMU; heterogeneous platforms including RC cars, legged robots, aerial robots, and wheeled robots; and extreme obscurant conditions such as dense fog, dust, smoke, and heavy snow.
We develop accuracy and robustness evaluation tracks for SLAM and introduced novel robustness metrics. Comprehensive studies are performed, revealing new observations, challenges, and opportunities for future research.
All-weather Environments
- Spanning 5 years, SubT-MRS comprises over 2000 hours and 300 kilometers of accurately measured trajectories. SubT-MRS is the first real-world dataset that specifically addresses failure scenarios of SLAM by incorporating a variety of degraded conditions, multiple robotic platforms, and diverse sets of multimodal sensors.
High-precision Ground Truth Maps
To evaluate the actual SLAM performance running on robots, we propose a robustness metric, which, to the best of our knowledge, is the first metric that can evaluate the reliability, safety, and resilience of SLAM systems.
