DIRAC - Distributed Infrastructure with Remote Agent Control
N. Brook, A. Bogdanchikov, A. Buckley, J. Closier, U. Egede, M. Frank, D. Galli, M. Gandelman, V. Garonne, C. Gaspar, R. Graciani Diaz, K. Harrison, E. van Herwijnen, A. Khan, S. Klous, I. Korolko, G. Kuznetsov, F. Loverre, U. Marconi, J. P. Palacios, G. N. Patrick, A. Pickford, S. Ponce, V. Romanovski, J. J. Saborido, M. Schmelling, A. Soroko, A. Tsaregorodtsev, V. Vagnoni, A. Washbrook
Abstract
This paper describes DIRAC, the LHCb Monte Carlo production system. DIRAC has a client/server architecture based on: Compute elements distributed among the collaborating institutes; Databases for production management, bookkeeping (the metadata catalogue) and software configuration; Monitoring and cataloguing services for updating and accessing the databases. Locally installed software agents implemented in Python monitor the local batch queue, interrogate the production database for any outstanding production requests using the XML-RPC protocol and initiate the job submission. The agent checks and, if necessary, installs any required software automatically. After the job has processed the events, the agent transfers the output data and updates the metadata catalogue. DIRAC has been successfully installed at 18 collaborating institutes, including the DataGRID, and has been used in recent Physics Data Challenges. In the near to medium term future we must use a mixed environment with different types of grid middleware or no middleware. We describe how this flexibility has been achieved and how ubiquitously available grid middleware would improve DIRAC.
Create a lesson
Related papers
Workload-Static Optical Circuit Switching For Scalable LLM Inference Decode
Maximilian Thürmer, Nicolas Scotto di Perto, Eloise Stein et al.
Characterizing Serverless Workloads: A Large-Scale Analysis of Function-as-a-Service Platforms
Saman Akbari, Yassine Lazreg, Mohammadreza Malekabbasi et al.
ForgeKernel: Self-Evolving Kernels Catching and Surpassing Experts and Agents
Yaojian Chen, Dazhi Peng, Yiyun Zheng et al.
Local Distributed Computation under Crash Failures
Diana Ghinea, Fabian Kuhn, Chen-Da Liu-Zhang et al.
TapeSAC: Automated, Pareto-Optimal Fine-Grained Memory Management for Large Model Training
Kaihui Ren, Yifan Zhao, Zejian Wang et al.
Accelerating Offloading-Based MoE Inference by Interleaving Concurrent Requests
Hongseung Yu, In Gim, Kyunghan Lee et al.