ReMoSPLAT: Reactive Mobile Manipulation Control on a Gaussian Splat

Abstract

Reactive control can gracefully coordinate the motion of the base and the arm of a mobile manipulator. However, incorporating an accurate representation of the environment to avoid obstacles without involving costly planning remains a challenge. In this work, we present ReMoSPLAT, a reactive controller based on a quadratic program formulation for mobile manipulation that leverages a Gaussian Splat representation for collision avoidance. By integrating additional constraints and costs into the optimisation formulation, a mobile manipulator platform can reach its intended end effector pose while avoiding obstacles, even in cluttered scenes. We investigate the trade-offs of two methods for efficiently calculating robot-obstacle distances, comparing a purely geometric approach with a rasterisation-based approach. Our simulation experiments on both synthetic and real-world scans demonstrate the feasibility of the proposed method, achieving performance comparable to controllers that rely on perfect ground-truth information. We further validate the approach on a real robot platform more details: https://remosplat.github.io