Design and Evaluation of Mechanisms for a Multicomputer Object Store
Lex Weaver
Abstract
Multicomputers have traditionally been viewed as powerful compute engines. It is from this perspective that they have been applied to various problems in order to achieve significant performance gains. There are many applications for which this compute intensive approach is only a partial solution. CAD, virtual reality, simulation, document management and analysis all require timely access to large amounts of data. This thesis investigates the use of the object store paradigm to harness the large distributed memories found on multicomputers. The design, implementation, and evaluation of a distributed object server on the Fujitsu AP1000 is described. The performance of the distributed object server under example applications, mainly physical simulation problems, is used to evaluate solutions to the problems of client space recovery, object migration, and coherence maintenance. The distributed object server follows the client-server model, allows object replication, and uses binary semaphores as a concurrency control measure. Instrumentation of the server under these applications supports several conclusions: client space recovery should be dynamically controlled by the application, predictively prefetching object replicas yields benefits in restricted circumstances, object migration by storage unit (segment) is not generally suitable where there are many objects per storage unit, and binary semaphores are an expensive concurrency control measure in this environment.
Create a lesson
Related papers
PixelFlow: Token-Level Workload Management for Efficient Distributed DiT Serving
Zhexiang Zhang, Minchen Yu, Yifan Sun et al.
A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum
Ignjat Karanovic, Pantelis A. Frangoudis, Ivan Čilić et al.
Accelerating Sharded Data Parallelism at Scale with Federated Learning
Gianluca Mittone, Marco Aldinucci
Distributed Edge Inference: an Experimental Study on Multiview Detection
Gianluca Mittone, Giulio Malenza, Marco Aldinucci et al.
P-GADMM: Parallel Group-Based ADMM for Asynchronous Optimization in Heterogeneous Edge Networks
Gaiguo Wei, Qingying Zhang, Heqiang Wang et al.
VERA: Reinforcement Learning for Dynamic Memory Scaling of HPC Workloads in Kubernetes
Ade Pramono, Jie Ren, Ivy Peng