Monday, October 17, 2011

PNUTS: Yahoo's Hosted Data Serving Platform

PNUTS is a distributed database designed by Yahoo especially for web applications that require scalability, high availability/fault tolerance, and low response time, while tolerating relaxed consistency guarantees. The main idea here is that all high-latency operations are performed asynchronously, while supporting record-level replication/mastering. A pub/sub message system is used instead of a traditional database log to coordinate replication. Data is automatically horizontally partitioned into groups of records called tablets.  These tablets are small (several can be a stored on each node) and can automatically be rearranged among nodes for load balancing.The system is implemented as a single centrally-managed hosted service, which is queried by applications.

Different parts of applications often require varying consistency guarantees -- slightly stale data might often be acceptable, while the latest version is needed in other locations (e.g. to provide show a user that an update has been made). PNUTS uses a per-record timeline consistency model, which guarantees that replicas of a record all write updates in the same order. Applications can then specify what kind of consistency they need on a per-record basis, using less expensive consistency requirements where possible (where stale data is acceptable).

It looks like PNUTS does a good job of balancing trade-offs between differing consistency requirements and latency in a geographically distributed database system by letting applications explicitly set their per-record requirements. The per-record replication and timeline consistency could be problematic when a query depends on the mutual consistency of several records (though this is something that the COPS paper addresses). That said, I think this approach of letting applications decide what kinds of consistency guarantees to require--and when--makes sense and should be very influential over the next decade as web services continue to geographically distribute their data across the world, while trying to maintain stringent latency policies.

No comments:

Post a Comment