Toward Foundation Detection Models via Example-Based Grounding in EM-DETR
Sheethal Bhat, Bogdan Georgescu, Awais Mansoor, Florin C. Ghesu, Sasa Grbic, Andreas Maier
Abstract
DeepLearning (DL)-based Computer-Aided Diagnostic (CAD) systems have shown great promise in supporting clinicians, but their performance heavily depends on the availability of large, well-annotated datasets. In medical lesion detection, the scarcity of comprehensive, stan dardized annotations and the heterogeneity of existing datasets limit the development of robust foundation models. In this work, we introduce a novel approach to building a multi-domain foundation model for le sion detection by integrating diverse medical image datasets annotated for both anatomical structures and abnormalities. We first propose a student-teacher training strategy to effectively combine datasets with heterogeneous label spaces, mitigating catastrophic forgetting, and im proving feature learning. In addition, building on our previously pro posed example-based EM-DETR framework, we adapt it to jointly learn from multiple domain-specific datasets, enabling fine-grained, example driven detection across modalities. Our model achieves state-of-the-art (SOTA) results, with image-level AUC scores exceeding previous SOTA by 1% point on four key findings in CXRs. We achieve a leading mAP50 in CXRs and mammography, surpassing EM-DETR trained on a single dataset by 2-3% points. This approach paves the way for scalable foundation models in medical imaging that leverage heterogeneous data while maintaining robust and generalizable performance.
Create a lesson
Related papers
Posterior Matching Distillation
Hidir Yesiltepe, Julius Berner, Arash Vahdat et al.
RL Post-Training in Diffusion Transformers Is Activation Steering
Tuna Han Salih Meral, Kaan Oktay, Alec Helbling et al.
One Prior, Two Dynamics: Repurposing Video Diffusion Transformers for Self-Supervised Latent Action Learning
Yuxin Jiang, Wenqiang Sun, Jiaqi Li et al.
ScaleWorld: High-Resolution World Generation through Shared Prediction and Refinement
Yansong Qu, Wesker Zheng, Quanjian Song et al.
Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth
Peiyun Hu, Marko Mihajlovic, Lyndon Duong et al.
Coda-DaVinci: Reimagining Video Generation with Code2Video
Yiyan Ji, Qixun Wang, Yang Shi et al.