C2-Routing: Complementary Cropping and Compression for Efficient Vision-Language Inference
Guorui Lu, Qinyu Chen
Abstract
Visual Question Answering (VQA) imposes substantial computational overhead on vision-language models (VLMs), motivating model-agnostic input reduction methods that can be readily applied across diverse VLMs, including in black-box settings. Simply enforcing a visual-token cap reduces computation, but it can discard information needed to answer the question. A single reduction strategy also struggles to accommodate different scenarios. Our key observation is that cropping and compression are complementary: cropping highlights small targets and local details, whereas whole-image compression better preserves context for large targets or distributed evidence. Based on this observation, we propose C2-Routing, which adaptively selects cropping or compression according to the input image and question. Since both strategies may still discard key information, we further design a safety gate that uses the predicted joint correctness distribution under original and reduced inputs to guide fallback and protect accuracy. To accommodate consumer GPU resource constraints, all training and evaluation experiments run on RTX 3090 GPUs. To measure gains beyond token capping, we use full-image baselines with the same cap of 1,024 visual tokens. We conduct extensive evaluation on three VLMs and 18 datasets spanning local details, global scenes, scene text, and documents. With the visual budget already capped, C2-Routing further reduces end-to-end FLOPs by 36.53% and visual tokens by 42.36% on average, with a mean relative task-score decrease of only 0.77%.
Create a lesson
Related papers
Posterior Matching Distillation
Hidir Yesiltepe, Julius Berner, Arash Vahdat et al.
RL Post-Training in Diffusion Transformers Is Activation Steering
Tuna Han Salih Meral, Kaan Oktay, Alec Helbling et al.
One Prior, Two Dynamics: Repurposing Video Diffusion Transformers for Self-Supervised Latent Action Learning
Yuxin Jiang, Wenqiang Sun, Jiaqi Li et al.
ScaleWorld: High-Resolution World Generation through Shared Prediction and Refinement
Yansong Qu, Wesker Zheng, Quanjian Song et al.
Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth
Peiyun Hu, Marko Mihajlovic, Lyndon Duong et al.
Coda-DaVinci: Reimagining Video Generation with Code2Video
Yiyan Ji, Qixun Wang, Yang Shi et al.