POOL File Catalog, Collection and Metadata Components
C. Cioffi, S. Eckmann, M. Girone, J. Hrivnac, D. Malon, H. Schmuecker, A. Vaniachine, J. Wojcieszuk, Z. Xie
Abstract
The POOL project is the common persistency framework for the LHC experiments to store petabytes of experiment data and metadata in a distributed and grid enabled way. POOL is a hybrid event store consisting of a data streaming layer and a relational layer. This paper describes the design of file catalog, collection and metadata components which are not part of the data streaming layer of POOL and outlines how POOL aims to provide transparent and efficient data access for a wide range of environments and use cases - ranging from a large production site down to a single disconnected laptops. The file catalog is the central POOL component translating logical data references to physical data files in a grid environment. POOL collections with their associated metadata provide an abstract way of accessing experiment data via their logical grouping into sets of related data objects.
Create a lesson
Related papers
Navigating Runtime Dynamism in Semantic Query Serving
Hengrui Zhang, Xupeng Miao
SQLoom: Set-Contrastive Learning of Syntactic, Semantic, and Logical Dependencies for Text-to-SQL
Hao Fu, Degang Sun, Wan Wei et al.
Stop Handcrafting Your Engine! GenDB-V: Towards AI-Generated and Formally Verified Database Systems
Jiale Lao, Immanuel Trummer
Recast: Refinement-Centered Approximate Nearest Neighbor Search Optimized for Storage
Zehai Yang, Fengyi Tao, Wenzhuo Dong et al.
USDB: Scaling Storage for Next-Generation Recommender Systems at ByteDance
Xiaoran Zhu, Jieming Cai, Weiping Yu et al.
Mixed-Type Relational Database Synthesis with Graph-Prompted Language Model
Yuxiang Sun, Mengshi Chen, Jianwei Wang et al.