DeRep: Learning Depth Representations for End-to-End Navigation in Cluttered Environments
Hanyu Jin, Kenji Shimada
Abstract
End-to-end navigation in cluttered environments relies on compact visual representations to support collision avoidance and goal-directed motion. Existing methods typically learn these representations jointly with the policy or through observation reconstruction, which does not explicitly specify what information the representation should preserve. Moreover, they are usually evaluated with a single downstream policy learner, although the benefit of a representation may depend on how the policy is learned. It therefore remains unclear how different representation-learning objectives affect downstream navigation performance and robustness. This work proposes DeRep, a navigation-oriented representation-learning method that combines local bird's-eye-view reachability supervision with depth reconstruction. To compare representation objectives under controlled conditions, we pretrain encoders with a common architecture and freeze them during downstream learning. We evaluate the representations using a linear policy probe, flow-matching imitation learning, and reinforcement learning with PPO. DeRep achieves the lowest linear policy probe prediction error and outperforms reconstruction alone across all six tested PPO conditions. In imitation learning, both reachability-based objectives achieve higher mean success rates than reconstruction alone, with similar mean success rates between the reachability-only and joint objectives. These results demonstrate consistent benefits of incorporating reachability supervision into depth representation pretraining across different downstream learning methods and evaluation conditions. We further deploy the PPO policy with the frozen DeRep encoder in Gazebo and on a physical quadrotor without fine-tuning, demonstrating its capability for zero-shot sim-to-sim and sim-to-real transfer.
Create a lesson
Related papers
AgentRoboCritic: Understanding the Capability of Frontier Agents on Direct Robot Control
Yiyang Du, Yuxuan Kuang, Qitao Zhao et al.
A Tutorial on Complementarity-Based Contact Models for Robotics
Onur Beker, A. René Geist, Michael Posa et al.
PGX: Physically Grounded Execution for Long-Horizon Humanoid Loco-Manipulation
Kai Lu, Chenyang Ma, Ioannis Havoutis
PhyDEX: A Benchmark for Physical Reasoning in Vision-Language-Action Models via Dynamic Extrinsic Dexterity
Hao Vo, Taisei Hanyu, Yuki Ikebe et al.
ACE-Brain-0.7: Learning Execution-Grounded Reasoning for Embodied Agents
Yinhe Wang, Shijun Shi, Feng Pan et al.
JARVIS: An Agent That Designs, Builds, and Tests Circuit Boards on Its Own
Shibo Zhou, Tianqi Hong, Zheng Yan et al.