Line Multiview Ideals

Abstract

We study the following problem in computer vision from the perspective of algebraic geometry: Using $m$ pinhole cameras we take $m$ pictures of a line in $\mathbb P^3$. This produces $m$ lines in $\mathbb P^2$ and the question is which $m$-tuples of lines can arise that way. We are interested in polynomial equations and therefore study the complex Zariski closure of all such tuples of lines. The resulting algebraic variety is a subvariety of $(\mathbb P^2)^m$ and is called line multiview variety. In this article, we study its ideal. We show that for generic cameras the ideal is generated by $3\times 3$-minors of a specific matrix. We also compute Gr\"obner bases and discuss to what extent our results carry over to the non-generic case.