Optimizing Tail Latency in Commodity Datacenters using Forward Error Correction

Abstract

Long tail latency of short flows (or messages) greatly affects user-facing applications in datacenters. Prior solutions to the problem introduce significant implementation complexities, such as global state monitoring, complex network control, or non-trivial switch modifications. While promising superior performance, they are hard to implement in practice. This paper presents CloudBurst, a simple, effective yet readily deployable solution achieving similar or even better results without introducing the above complexities. At its core, CloudBurst explores forward error correction (FEC) over multipath - it proactively spreads FEC-coded packets generated from messages over multipath in parallel, and recovers them with the first few arriving ones. As a result, CloudBurst is able to obliviously exploit underutilized paths, thus achieving low tail latency. We have implemented CloudBurst as a user-space library, and deployed it on a testbed with commodity switches. Our testbed and simulation experiments show the superior performance of CloudBurst. For example, CloudBurst achieves 63.69% and 60.06% reduction in 99th percentile message/flow completion time (FCT) compared to DCTCP and PIAS, respectively.