This is the fifth and final episode that we’re re-releasing from back when we told Chris’s personal story of starting Viathan, which was a software startup built to create one of the first internet scale databases. In this episode, we compare Viathan’s pre-cloud architecture to that of DynamoDB. They’re less different than you might expect!

Mobycast

Mobycast.fm

Replay of Ep 43 - The Birth of NoSQL and DynamoDb – Part 5

APR 15, 202042 MIN
Mobycast

Replay of Ep 43 - The Birth of NoSQL and DynamoDb – Part 5

APR 15, 202042 MIN

Description

Show DetailsJon Christensen and Chris Hickman of Kelsus and Rich Staats of Secret Stache conclude their series on the birth of NoSQL and DynamoDB. They compare the NoSQL database, Leviathan, created by Chris’s startup in the late 1990s to today’s DynamoDB. A lot of things haven’t changed, even though technology has evolved. It’s cyclical. There are patterns and problems that continue to dominate.  Some of the highlights of the show include:
  • Reason for Creation of NoSQL Database: How to scale database with Internet-scale applications to have a virtual pool of infinite storage that can be scaled out
  • Main Architecture Components of Leviathan:
    • API client
    • Update distributor (UD)
    • Base server (storage node)
    • Shepherd (housekeeping management system)  
  • Additional core components included smart IP and storage abstraction layer (SAL)
  • Leviathan mostly used C code and minimal Java code to support users
  • Big difference between DynamoDB and Leviathan is request router and partition metadata system living on the server vs. living on the edge
  • Leviathan was a closed system with an instance for every network or data center; not designed to run as a software as a service, like DynamoDB
  • Leviathan was strongly consistent, unlike DynamoDB’s eventually consistent model
  • Definition and Different Types of Transactions
  • Shepherd was used to identify and address consistency, synchronous, and timing issues 
  • Rather than using a file system, Leviathan used relational databases 
Links and ResourcesDynamoDBMicrosoft SQLOracle DBAWS IoT GreengrassKelsusSecret Stache Media Quotes:“We had the same kind of problems that DynamoDB had - how do you scale your database dealing with Internet-scale applications and have this virtual pool of infinite storage that can be scaled out.” Chris Hickman “This system and this technology went through many iterations.” Chris Hickman “You can’t have a 100% consistent state across everything. It’s just impossible. How do you do the right thing?” Chris Hickman “The big difference between DynamoDB and Leviathan...is the request router and partition metadata system living on the server vs. living out at the edge.” Jon Christen