AliEn Resource Brokers
Pablo Saiz, Predrag Buncic, Andreas J. Peters
Abstract
AliEn (ALICE Environment) is a lightweight GRID framework developed by the Alice Collaboration. When the experiment starts running, it will collect data at a rate of approximately 2 PB per year, producing O(109) files per year. All these files, including all simulated events generated during the preparation phase of the experiment, must be accounted and reliably tracked in the GRID environment. The backbone of AliEn is a distributed file catalogue, which associates universal logical file name to physical file names for each dataset and provides transparent access to datasets independently of physical location. The file replication and transport is carried out under the control of the File Transport Broker. In addition, the file catalogue maintains information about every job running in the system. The jobs are distributed by the Job Resource Broker that is implemented using a simplified pull (as opposed to traditional push) architecture. This paper describes the Job and File Transport Resource Brokers and shows that a similar architecture can be applied to solve both problems.
Create a lesson
Related papers
GC-MoE: Overlapping Expert-Parallel Token Dispatch with FP8/FP4 Grouped GEMM via CUDA Green Contexts
Yong Li, Kan Liu, Tao Lan
When the Consumer Is an Agent: Execution-Grounded Service-Level Agreements
Michal Orzechowski, Renata G. Slota, Jacek Kitowski
PhotonZip: Lifecycle-Aware Compression for HDF5-Based Synchrotron Workflows
Haoxu Li, Wenjing Huang, Chengyu Liu et al.
EpiLoRA: Fusing Many LoRA Adapters into Low-Precision GEMMs for Serving and RL
Hieu Tran, Hong Yu
INSIGHT-HPC: An Interpretable System Intelligence for Global HPC Telemetry
Rupak Kadel, Jie Li, Yong Chen
DynSL: Resource-Adaptive Model Training across Heterogeneous Edge Devices
Boris Radovič, Istemi Ekin Akkus, Marco Canini et al.