Off-road navigation is critical for a wide range of field robotics applications from planetary exploration to disaster response. However, it remains a longstanding challenge due to unstructured environments and the inherently complex terrain-vehicle interactions. Traditional physics-based methods struggle to accurately capture the nonlinear dynamics underlying these interactions, while purely data-driven approaches often overfit to specific motion patterns, vehicle geometries, or platforms, limiting their generalization in diverse, real-world scenarios. To address these limitations, we introduce AnyNav, a vision-based friction estimation and navigation framework grounded in neuro-symbolic principles. Our approach integrates neural networks for visual perception with symbolic physical models for reasoning about terrain-vehicle dynamics. To enable self-supervised learning in real-world settings, we adopt the imperative learning paradigm, employing bilevel optimization to train the friction network through physics-based optimization. This explicit incorporation of physical reasoning substantially enhances generalization across terrains, vehicle types, and operational conditions. Leveraging the predicted friction coefficients, we further develop a physics-informed navigation system capable of generating physically feasible, time-efficient paths together with corresponding speed profiles. We demonstrate that AnyNav seamlessly transfers from simulation to real-world robotic platforms, exhibiting strong robustness across different four-wheeled vehicles and diverse off-road environments.
Method Overview
AnyNav comprises a neuro-symbolic friction learning module and a physics-informed navigation module. The neuro-symbolic module has a neural network to predict friction coefficients from visual inputs, guided by symbolic physics reasoning for self-supervised training. The inputs are captured by camera and LiDAR before entering a terrain patch, while training occurs afterward using the dynamics observed on that patch. The navigation module builds terrain property maps online and leverages the predicted friction information to generate physically feasible and efficient path and speed profiles for off-road driving.
Demonstrations
Let’s consider a navigation task as an example. Given a start point and a target point on the map, which path is optimal and at what speed should we travel?
We compare AnyNav (orange) with a planner that does not account for friction prediction (blue). As shown, AnyNav selects a less slippery dirt path to ascend the hill, while the other planner opts for a shorter but highly slippery icy route, as it lacks friction awareness. To evaluate the two plans, we execute them in simulation. Both use the same vehicle, controlled by a PD controller to follow the planned trajectories. AnyNav successfully guides the vehicle to its destination, whereas the other planner fails to climb the icy hill.
We also evaluate AnyNav in a more visually realistic island environment, using randomly selected start and target points. This showcases the robustness and reliability of our system in navigating complex off-road terrains with physics-infused planning.
We deployed AnyNav on two wheeled robots including a Rover and a Racer. A real-time mapping system is developed for terrain reconstruction and analysis. Using the RViz interface, users can select points of interest, after which our physics-informed planner generates trajectories for traversal. The system efficiently guides the vehicle to its targets by avoiding obstacles and prioritizing routes with higher grip.
