Federated Learning on the American Science Cloud using APPFL
Zilinghan Li, Abhijit Chunduru, Harinarayan Krishnan, Eric Chagnon, Peter Nugent, Kibaek Kim, Ravi Madduri
Abstract
The American Science Cloud (AmSC), established under the Genesis Mission of the U.S. Department of Energy (DOE), aims to integrate DOE high-performance computing systems, experimental facilities, and data resources into a single, coordinated, AI-driven discovery platform. AmSC's early services focus on curated artifacts, such as gated inference access to hosted models, experiment tracking, and function execution across computing facilities. However, what these services lack is a means to train a model across organizational boundaries where data cannot be centralized due to policy, privacy, or scale. This is, by definition, a use case for federated learning (FL) and a growing class of scientific AI. In this paper, we show that this gap can be bridged by deploying the orchestration logic of the Advanced Privacy-Preserving Federated Learning (APPFL) framework as a scalable cloud service on top of the primitives AmSC already provides: project-scoped authentication that supports secure and reliable federation membership, function execution that drives distributed training at each site, experiment tracking that records round-level performance, and finally, the model-hosting and inference infrastructure that can be leveraged to distribute the federated trained models to authorized participants. We argue that offering federated computing as an important AmSC service would unlock privacy-constrained scientific collaborations, enabling public-private partnerships in model building while exercising and enhancing the platform's own federated infrastructure.
Create a lesson
Related papers
AceSpec: An Asymmetric Edge-Cloud Collaborative Framework for Communication-Efficient LLM Inference
Yida Zhang, Zhiyong Gao, Shuaibing Yue et al.
Towards Global Federated Genome-Wide Association Meta-Analysis Using GA4GH TES
Abhijit Chunduru, Matthew Joel, Zilinghan Li et al.
MeanField Surrogate Modeling for Scalable Runtime Scheduling of Concurrent Heterogeneous AI Inference on Shared GPUs
Youssef Ennouri, Soonhoi Ha
RT-HiSS: Ray Tracing Accelerated High Dimensional Vector Similarity Searches
Revanth Reddy Munugala, Michael Gowanlock
CREDIT: Cost-guided Reduction-reuse with Efficient DSMEM Inter-CTA Tiling
Zhengxiong Li, Tsung-Wei Huang, Umit Ogras
Scaling Inference Prefill with High-Radix Photonic Interconnects
Arulselvan Madhavan, Peter Carson, Taylor Groves et al.