The Spectrum Strikes Back: Infrared POV Attacks on Traffic Sign Classification

Abstract

Traffic sign classification is a crucial task for autonomous vehicles, and numerous attacks against it have been identified. A majority of physical adversarial attacks involve attaching patches to traffic signs or projecting perturbations on them. While they demonstrate high effectiveness, they are perceptible to humans. At the same time, light-based attacks outside the human visible spectrum are known but have limitations in their dynamic adaptability. We propose a persistence-of-vision-based attack that operates in the near-infrared light spectrum. With the possibility of showing dynamic, remotely triggered content, this allows a stealthy physical adversarial attack against traffic sign classification. By identifying the optimal position through digital simulation, we conduct extensive real-world evaluations using two different traffic signs, 12 machine learning models from different families, multiple distances up to 20 meters, and varying illumination conditions. Our evaluation shows high attack success rates across our test scenarios. We propose near-infrared cutoff filters and a software-based detection mechanism as defenses, and tackle limitations of the near-infrared persistence of vision display by prototyping a human-visible RGB version of it.