Dynamo (storage system)

Dynamo is the name given to a set of techniques that when taken together can form a highly available key-value structured storage system[1] or a distributed data store.[2] It has properties of both databases and distributed hash tables (DHTs). Amazon DynamoDB is "built on the principles of Dynamo" [3] (apparently with some improvements) and is a hosted service within the AWS infrastructure. It was created to help address some scalability issues that Amazon.com's website experienced during the holiday season of 2004.[4] By 2007 it was used in Amazon Web Services, such as its Simple Storage Service (S3).[5]


Techniques used

Problem Technique Advantage
Dataset partitioning Consistent Hashing Incremental, possibly linear scalability in proportion to the number of collaborating nodes.
Highly available writes Vector Clock or Dotted-Version-Vector Sets, reconciliation during reads Version size is decoupled from update rates.
Handling temporary failures Sloppy Quorum and Hinted Handoff Provides high availability and durability guarantee when some of the replicas are not available.
Recovering from permanent failures anti-entropy using Merkle tree Can be used to identify differences between replica owners and synchronize divergent replicas pro-actively.
Membership and failure detection gossip-based membership protocol and failure detection Avoids having a centralized registry for storing membership and node liveness information, preserving symmetry.

Implementation History

Amazon published the paper on Dynamo, but never released their implementation. The index layer of Amazon S3 implements and extends many of the core features of a Dynamo. Since then, several implementations have been created based on the paper. The paper also inspired many other NoSQL database implementations.[6] Here are some projects that implement it or were inspired by it.

See also


  1. Dynamo: Amazon’s Highly Available Key-value Store, SOSP 2007
  2. Decandia, G.; Hastorun, D.; Jampani, M.; Kakulapati, G.; Lakshman, A.; Pilchin, A.; Sivasubramanian, S.; Vosshall, P.; Vogels, W. (2007). "Dynamo". Proceedings of twenty-first ACM SIGOPS symposium on Operating systems principles - SOSP '07. p. 205. doi:10.1145/1294261.1294281. ISBN 9781595935915.
  3. http://www.allthingsdistributed.com/2012/01/amazon-dynamodb.html
  4. http://readwrite.com/2012/01/18/amazon-enters-the-nosql-market
  5. Werner Vogels (October 2, 2007). "Amazon's Dynamo". Retrieved September 21, 2016.
  6. http://readwrite.com/2012/01/18/amazon-enters-the-nosql-market "launched a thousand NoSQL databases"

External links

This article is issued from Wikipedia - version of the 9/21/2016. The text is available under the Creative Commons Attribution/Share Alike but additional terms may apply for the media files.