Prototypical Few-Shot Medical Image Semantic Segmentation with Background Fusion

Abstract

Few-shot Semantic Segmentation (FSS) aims to adapt a pre-trained model to new classes with as few as a single labeled training sample per class. The existing prototypical work used in natural image scenarios biasedly focus on capturing foreground's discrimination while employing a simplistic representation for background, grounded on the inherent observation separation between foreground and background. However, a frequency spectrum entropy analysis suggests that this paradigm is not applicable to medical images where the foreground and background share numerous visual features, necessitating a more detailed description for the background. In this paper, we present a new Background-fused prototype (Bro) approach for FSS in medical images. Instead of identifying a commonality of background subjects in the support image, Bro fuses this background to discriminative prototypes, with two pivot designs. Specifically, Feature Similarity Calibration (FeaC) initially reduces noise in the support image by employing feature cross-attention with the query image. Subsequently, Hierarchical Channel-Adversarial Attention (HiCA) merges the background into comprehensive prototypes. We achieve this by a channel groups-based attention mechanism, where an adversarial Mean-Offset structure encourages a coarse-to-fine fusion. Designed as a generic plug-in, our Bro can be seamlessly integrated with existing FSS models. Extensive experiments validate the specificity of the background in medical images and the efficacy of Bro in enhancing the performance of previous FSS models on standard benchmarks.