Performing work efficiently in the presence of faults
Cynthia Dwork, Joseph Y. Halpern, O. Waarts
Abstract
We consider a system of t synchronous processes that communicate only by sending messages to one another, and that together must perform n independent units of work. Processes may fail by crashing; we want to guarantee that in every execution of the protocol in which at least one process survives, all n units of work will be performed. We consider three parameters: the number of messages sent, the total number of units of work performed (including multiplicities), and time. We present three protocols for solving the problem. All three are work-optimal, doing O(n+t) work. The first has moderate costs in the remaining two parameters, sending O(tt) messages, and taking O(n+t) time. This protocol can be easily modified to run in any completely asynchronous system equipped with a failure detection mechanism. The second sends only O(t logt) messages, but its running time is large (exponential in n and t). The third is essentially time-optimal in the (usual) case in which there are no failures, and its time complexity degrades gracefully as the number of failures increases.
Create a lesson
Related papers
Understanding NVIDIA Grace-Blackwell Unified Memory Through Placement-Controlled Microbenchmarks
Nathan Graddon, Aaron Jarmusch, Sunita Chandrasekaran
TrimSO(2): Fast and Memory-Efficient Training of Equivariant Interatomic Potentials
Chen Wang, Yuanchang Zhou, Guangming Tan et al.
JointServe: Serving Multi-Model Agents on Memory-Constrained GPUs
Daijin Hu
The Cost of Context: Profiling the Energy Footprint of Input Tokens in Large Language Models
Boris Ruf, Marcin Detyniecki
Scaling LLM Inference via Parallelism: Throughput-Latency Trade-off
Javad Mirzaei, Jeebak Mitra
Parallel Sparse Outputs for Tensor Compilers
Adrian Gushin, Pau Sum, Willow Ahrens