Test-Time Adaptation via Cache Personalization for Facial Expression Recognition in Videos

Abstract

Facial expression recognition (FER) in videos requires model personalization to capture considerable variations across subjects. Vision-language models (VLMs) offer strong transfer to downstream tasks through image-text alignment, but their performance can degrade under inter-subject distribution shifts. Test-time adaptation (TTA) can mitigate this challenge. However, most state-of-the-art TTA methods rely on unsupervised parameter optimization, introducing computational overhead that is impractical in many real-world applications. This paper introduces TTA through Cache Personalization (TTA-CaP), a cache-based TTA method that enables cost-effective, gradient-free personalization of VLMs for video FER. Prior cache-based TTA methods rely solely on dynamic memories that store test samples, which can accumulate errors and drift because of noisy pseudo-labels. TTA-CaP instead introduces three complementary caches: a personalized static cache constructed through feature-statistics matching, a positive target cache that accumulates reliable subject-specific samples, and a negative target cache that stores low-confidence cases as negative samples. To prevent target-cache corruption, a tri-gate mechanism controls cache updates based on temporal stability, confidence, and consistency with the personalized static cache. Together, these caches provide complementary, subject-matched positive and negative evidence for robust online personalization. TTA-CaP further refines predictions by fusing embeddings, yielding representations that support temporally stable video-level predictions. Experiments on BioVid, StressID, and BAH show that TTA-CaP outperforms state-of-the-art TTA methods under subject-specific and environmental shifts while maintaining low computational and memory overhead. Our code is publicly available at https://github.com/MasoumehSharafi/TTA-CaP.