Distributed Transactions for Google App Engine: Optimistic Distributed Transactions built upon Local Multi-Version Concurrency Control

Abstract

Massively scalable web applications encounter a fundamental tension in computing between "performance" and "correctness": performance is often addressed by using a large and therefore distributed machine where programs are multi-threaded and interruptible, whereas correctness requires data invariants to be maintained with certainty. A solution to this problem is "transactions" [Gray-Reuter]. Some distributed systems such as Google App Engine [http://code.google.com/appengine/docs/] provide transaction semantics but only for functions that access one of a set of predefined local regions of the database: a "Local Transaction" (LT) [http://code.google.com/appengine/docs/python/datastore/transactions.html]. To address this problem we give a "Distributed Transaction" (DT) algorithm which provides transaction semantics for functions that operate on any set of objects distributed across the machine. Our algorithm is in an "optimistic" [http://en.wikipedia.org/wiki/Optimistic_concurrency_control] style. We assume Sequential [Time-]Consistency [http://en.wikipedia.org/wiki/Sequential_consistency] for Local Transactions.