A Software Data Transport Framework for Trigger Applications on Clusters
Timm M. Steinbeck, Volker Lindenstruth, Heinz Tilsner
Abstract
In the future ALICE heavy ion experiment at CERN's Large Hadron Collider input data rates of up to 25 GB/s have to be handled by the High Level Trigger (HLT) system, which has to scale them down to at most 1.25 GB/s before being written to permanent storage. The HLT system that is being designed to cope with these data rates consists of a large PC cluster, up to the order of a 1000 nodes, connected by a fast network. For the software that will run on these nodes a flexible data transport and distribution software framework has been developed. This framework consists of a set of separate components, that can be connected via a common interface, allowing to construct different configurations for the HLT, that are even changeable at runtime. To ensure a fault-tolerant operation of the HLT, the framework includes a basic fail-over mechanism that will be further expanded in the future, utilizing the runtime reconnection feature of the framework's component interface. First performance tests show very promising results for the software, indicating that it can achieve an event rate for the data transport sufficiently high to satisfy ALICE's requirements.
Create a lesson
Related papers
PixelFlow: Token-Level Workload Management for Efficient Distributed DiT Serving
Zhexiang Zhang, Minchen Yu, Yifan Sun et al.
A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum
Ignjat Karanovic, Pantelis A. Frangoudis, Ivan Čilić et al.
Accelerating Sharded Data Parallelism at Scale with Federated Learning
Gianluca Mittone, Marco Aldinucci
Distributed Edge Inference: an Experimental Study on Multiview Detection
Gianluca Mittone, Giulio Malenza, Marco Aldinucci et al.
P-GADMM: Parallel Group-Based ADMM for Asynchronous Optimization in Heterogeneous Edge Networks
Gaiguo Wei, Qingying Zhang, Heqiang Wang et al.
VERA: Reinforcement Learning for Dynamic Memory Scaling of HPC Workloads in Kubernetes
Ade Pramono, Jie Ren, Ivy Peng