Off-road navigation is essential for a wide range of applications in field robotics such as planetary exploration and disaster response. However, it remains an unresolved challenge due to the unstructured environments and inherent complexity of terrain-vehicle interactions. Traditional physics-based methods struggle to accurately model the nonlinear dynamics of these interactions, while data-driven approaches often suffer from overfitting to specific motion patterns, vehicle sizes, and types, limiting their generalizability. To overcome these challenges, we introduce a vision-based friction estimation framework grounded in neuro-symbolic principles, integrating neural networks for visual perception with symbolic reasoning for physical modeling. This enables significantly improved generalization abilities through explicit physical reasoning incorporating the predicted friction. Additionally, we develop a physics-informed planner that leverages the learned friction coefficient to generate physically feasible and efficient paths, along with corresponding speed profiles. We refer to our approach as AnyNav and evaluate it in both simulation and real-world experiments, demonstrating its utility and robustness across various off-road scenarios and multiple types of four-wheeled vehicles. These results mark an important step toward developing neuro-symbolic spatial intelligence to reason about complex, unstructured environments and enable autonomous off-road navigation in challenging scenarios.

Method Overview

The AnyNav framework comprises a neuro-symbolic module and a planning module. The neuro-symbolic module has a neural network to predict friction coefficients from visual inputs, guided by symbolic physics reasoning for supervision. The planning module leverages friction knowledge to determine physically feasible and efficient paths and speeds for off-road navigation.

Demonstrations

Let’s consider a navigation task as an example. Given a start point and a target point on the map, which path is optimal and at what speed should we travel?

We compare AnyNav (orange) with a planner that does not account for friction prediction (blue). As shown, AnyNav selects a less slippery dirt path to ascend the hill, while the other planner opts for a shorter but highly slippery icy route, as it lacks friction awareness. To evaluate the two plans, we execute them in simulation. Both use the same vehicle, controlled by a PD controller to follow the planned trajectories. AnyNav successfully guides the vehicle to its destination, whereas the other planner fails to climb the icy hill.

We also evaluate AnyNav in a more visually realistic island environment, using randomly selected start and target points. This showcases the robustness and reliability of our system in navigating complex off-road terrains with physics-infused planning.