Co-occurrence-Aware Quadratic Assignment for Local Feature Matching in Simultaneous Localization and Mapping

Abstract

Local feature matching, which associates keypoints in two images as keypoint pairs, is fundamental to Visual Simultaneous Localization and Mapping (Visual SLAM). Nearest Neighbor (NN) search is commonly used for keypoint matching, but it has difficulty selecting correct keypoint pairs when multiple candidates have similar costs. To improve matching accuracy, this paper proposes a keypoint matching method that considers the pairwise co-occurrence of two keypoint pairs. The keypoint matching is formulated as a quadratic assignment problem, which is an NP-hard combinatorial optimization problem, making it difficult to solve quickly on conventional computers. Recently, Ising machines have been developed as computing devices capable of solving hard combinatorial optimization problems. Using a simulated bifurcation based Ising machine, the proposed method improved matching accuracy by approximately 8 percentage points over a conventional method on the HPatches dataset. Furthermore, we integrated the proposed method into ORB-SLAM3, a representative academic Visual SLAM system, and achieved a 3.78-fold improvement in absolute pose error (APE) and a 2.85-fold improvement in relative pose error (RPE) on the KITTI dataset scenes where multiple same shape objects are repeatedly arranged, which are challenging for accurate self-pose estimation by the original ORB-SLAM3.