How Much of SAR Vehicle Recognition Is the Vehicle? Background Shortcuts, Angle Gaps and Partial Libraries on MSTAR and ATRNet-STAR
Mousa Shaya
Abstract
Deep networks score near 99% on the MSTAR vehicle benchmark, and the newer ATRNet-STAR benchmark was built to be harder. We ask how much of these scores comes from the vehicle itself. With a modern network (ConvNeXt-Small, 3 seeds) we first reproduce the known MSTAR result: with the central 96x96 pixels zeroed out, accuracy is still 73.3% (chance 10%). The same shortcut is present in ATRNet-STAR, which was collected across several scenes: with the vehicle removed, accuracy is 63.3% on the 40-class SOC split (chance 2.5%). Under the official scene-change split, the background stops helping and starts hurting. Accuracy is 22.8% on full chips and 30.8% when only the vehicle is shown. A SAR foundation model, SARATR-X, leans on the background more than ConvNeXt does (+4 to +6 points with the vehicle removed, in every seed) and does not solve the scene change. We then hold the amount of training data fixed and vary the depression angles used for training. Accuracy is set mainly by the gap to the nearest training angle: 57% at 15 degrees, 39% at 30 degrees and 26% at 45 degrees. Adding more training angles does not help. Finally, when most classes have training images at the test angle and a few do not, the missing classes do worse than when no class has them: 9.0% instead of 22.4% at a 45-degree gap. Most of their images are assigned to known classes. We recommend reporting background-only and target-only controls, and the angle gap, next to any accuracy figure. Code is public.
Create a lesson
Related papers
Auditors Struggle to Catch Sabotage In End-to-End Transcripts of AI Safety Research
Keshav Shenoy, Anthony Zang, Eric Gan et al.
Chain-of-Thought Controllability Can Reduce Chain-of-Thought Monitorability
Zheng-Xin Yong, Benjamin Arnav, Miles Wang et al.
Right Reason, Wrong Advice: Evaluating Recourse in LLM Explanations of Denied Requests
Arunkumar V
SciResearch-Eval: Evaluating Scientific Judgment in Autonomous Experimental Agents
Brian T. Weston, Alondra Torres Contreras, Aldair E. Gongora
Procedure without Judgment: What Small Language Models Learn under an Exact Verifier
Arunkumar V, Manoranjan Gandhudi, Gangadharan G. R. et al.
Safe Resemblance, Silent Propagation: Limits of Text-Similarity Guardrails for AI Agent Safety
Charles Xiao