Reliable Data Storage in Distributed Hash Tables
Matthew Leslie
Abstract
Distributed Hash Tables offer a resilient lookup service for unstable distributed environments. Resilient data storage, however, requires additional data replication and maintenance algorithms. These algorithms can have an impact on both the performance and the scalability of the system. In this paper, we describe the goals and design space of these replication algorithms. We examine an existing replication algorithm, and present a new analysis of its reliability. We then present a new dynamic replication algorithm which can operate in unstable environments. We give several possible replica placement strategies for this algorithm, and show how they impact reliability and performance. Finally we compare all replication algorithms through simulation, showing quantitatively the difference between their bandwidth use, fault tolerance and performance.
Create a lesson
Related papers
PixelFlow: Token-Level Workload Management for Efficient Distributed DiT Serving
Zhexiang Zhang, Minchen Yu, Yifan Sun et al.
A Kubernetes-Native Request Router for Quality-Aware Inference Serving in the Computing Continuum
Ignjat Karanovic, Pantelis A. Frangoudis, Ivan Čilić et al.
Accelerating Sharded Data Parallelism at Scale with Federated Learning
Gianluca Mittone, Marco Aldinucci
Distributed Edge Inference: an Experimental Study on Multiview Detection
Gianluca Mittone, Giulio Malenza, Marco Aldinucci et al.
P-GADMM: Parallel Group-Based ADMM for Asynchronous Optimization in Heterogeneous Edge Networks
Gaiguo Wei, Qingying Zhang, Heqiang Wang et al.
VERA: Reinforcement Learning for Dynamic Memory Scaling of HPC Workloads in Kubernetes
Ade Pramono, Jie Ren, Ivy Peng