UniTAC: Universal Task-Aware Compression via Weighted Distortion Measures

Abstract

Lossy compression is conventionally driven by a task-agnostic distortion (e.g., MSE or MS-SSIM), yet in many emerging applications the receiver cares not about uniform fidelity but about a downstream task whose relevant content varies across the signal and evolves over time. We formulate task-aware compression as a weighted rate-distortion problem, in which a single codec is driven by a separable, per-component weighted distortion whose weights encode task importance and may depend on the source. We introduce task consistency, i.e., that minimizing the weighted distortion also minimizes the true task loss, and characterize when it holds: for linear tasks, the task loss admits a weighted-MSE form with signal-independent weights under suitable cross-term conditions, while for nonlinear tasks, an integrated-gradients analysis motivates separable task-aware weights. We show how task symmetry and irrelevance further constrain the admissible weights. Guided by this theory, we realize the weight-conditioned code in a single learned Vision Transformer (ViT) codec whose token-level attention natively consumes a per-component importance vector, so one fixed backbone is re-targeted at runtime, from universal (task-agnostic) to task-specialized operation, purely by swapping the injected weights, without retraining, while producing a single human-viewable reconstruction steered to the active task. On downstream face-analysis tasks, a single model reaches 91.4% accuracy at 0.034 bpp on a localized task, within 1.9% of a task-specific codec (93.3%) and well above a universal codec (76.9%). Such task-adaptive compression suits bandwidth-constrained perception systems, e.g., in Physical AI, where the active task drifts and per-task retraining is infeasible.