Pages

Monday, 14 October 2013

Which embedded key-value store to use? Or the curse of too much choice...

One of my first tasks at my new job (which I just started) was to create a service exposing a key-value store for storing BLOBs. The natural question was should I use our current database (H2) or something else.

The requirements were simple:
  1. embeddable Java solution (as we are making packaged software and don't really need the extra problems related to setting up a DB running in a separate process)
  2. license allowing us to use it in a commercial product (something along the lines of Apache License 2.0)
  3. not in-memory (sounded simply wrong for BLOBs)
  4. scalable (both horizontally and vertically)
  5. simple to set-up, use and well documented

Simple, right? Well the problem with simple problems is that there are too many ways to solve them. If you google for a "key value store" the number of possibilities will crush you. Numbers 1, 2 and 3 from the above list made the choice much easier (MongoDB, CouchDB, Hazelcast, Memchached, Riak, Redis, Berkeley DB, Neo4J etc. all failed at at least one of them) but still I was left with:

  • H2
  • Voldemort
  • OrientDB
  • Cassandra

And many others which I flat out rejected on a gut feeling (sorry, I won't be able to go to a customer and tell him we're using BananaDB).

If you're interested in some code or you'd like to see how the (very) basic configuration looks like I'm trying them out on my GitHub repo (although currently there's only very basic OrientDB setup commited). So how do the above compare?

OrientDB

OrientDB is an interesting NoSQL DBMS which stores data as documents but also allows direct relations between them like in graph databases. It also can work as a key-value engine! The creators say it's secure, web ready (by which they mean native JSON REST over HTTP support) and scalable. They claim to support schema-less mode, schema-full or a mix of both, ACID transaction and some other goodies.

  • What do you need to get started:
    • orientdb.config
    • according to the doc: orientdb-enterprise-*.jar && orientdb-server-*.jar (in practice you better use a build tool like Maven or Gradle since there are quite a few dependencies)
  • Ease of use: starting the server, creating/opening a DB and saving your first records is a matter of only few lines.
  • Documentation: sometimes lacking, at some points it's confusing whether it is still up-to-date or is it a page for an old version (mainly the google code docs). Overall passable.
  • Adoption: weak. The DB has already 3 years (since the initial release) but I failed to find a single well known company using it.
  • License: Apache 2.0
  • Protocol: HTTP and binary
  • Misc:
    • it uses a new interesting MVRB-Tree indexing algorithm that allows to store multiple values in a single tree node (opposite to only 1 value per node in RB Trees) and thanks to which both inserts and lookups are supposed to be fast
    • requires the new network node to have identical db before adding it to the network. I don't really see how this plays with the "scalable" part of OrientDB.
  • When to use: the basic case would be a very simple KV store. OrientDB fits here nicely as the MVRB indexing will make it fast. As a graph db on the other hand it goes well with any sort of data that is connected using complex relations (social networks anyone? but also applications handling geographical or routing data).

H2

Contrary to the other options I am considering here H2 is the only RDBMS. Frankly speaking the only reason I put it up here is because we already are using a H2 DB in our software and it might just turn out good enough. H2 seems like a fairly standard RDBMS which can be embedded in Java application. It supports a subset of SQL, mainly relies on it and JDBC, can be used both as in-memory and disk-based

  • What do you need to get started:
    • h2*.jar - then simply launch the server in 1 line of code and get a JDBC Data Source.
  • Ease of use: installation is one jar, starting up the whole thing is few lines of code. Rest is plain old JDBC or you can use an ORM, what can be easier?
  • Documentation: well written and maintained
  • Adoption: well it's no Cassandra but the list of projects using it still looks well
  • License: dual licensed under the MPL 1.1 (Mozilla Public License) or the (unmodified) EPL 1.0
  • Protocol: PostgreSQL server side network protocol
  • Misc:
    • this is still a RDBMS which means it has to be (or at least should be in theory) slower than the NoSQL solutions due to join and ACID support (though Orient DB also is supposed to have that one)
    • it seems to support only a very basic HA model
  • When to use: when you need a mature, fast DB that supports all the RDBMs goodness (mainly ACID, joins and normalization) and you don't necessarily need to handle very large amounts of unstructured data (and no, by very large I don't mean 1GB...)

Cassandra

Cassandra is a "BigTable meets Dynamo" Java database initially created at Facebook. By that I mean it has a Dynamo-like architecture and BigTable-like data model (columns and column families). It's main goals are high (linear) scalability across a large network (spanning over multiple data centres) of commodity devices. It provides a masterless architecture which also supports asynchronous replication of data. If you're looking for joins or subqueries you better keep on searching! Unless you're using batch Hadoop analysis.

  • What do you need to get started:
    • sources and a cassandra.yaml config file
  • Ease of use: here it's actually a bit more tricky than OrientDB. Setting up the CassandraDaemon and cleaning up after it seems a bit more complicated but there seem to already be open source helpers doing that for you.
  • Documentation: looking good, detailed and well maintained. Even if the documentation is not enough, there's plenty of information online about Cassandra because of all the hype it generated in the past few years.
  • Adoption: the list of companies using Cassandra seems to just go on and on.
  • License: Apache 2.0
  • Protocol: Thrift and binary CQL3
  • Misc:
    • writes faster than reads
    • easy to set the tradeoff between distribution and replication
    • great replication capabilities
  • When to use: since writes are here faster than reads it seems great for things like log storing applications or any other case of data analysis.

Voldemort

Voldemort is an open source implementation of Amazon's DynamoDB developed by the guys at LinkedIn (apparently one of the contributors actually worked on Dynamo) with scalability and high performance in mind. Apparently it gives linear scalability and one of the lowest latencies. To tell you the truth I was really looking forward to this solution and was a bit biased towards it but...

It seems the guys at LinkedIn don't like to couple their projects with Maven and they don't think that deploying their ant projects to a maven repository is a high priority. For most people it might not be a problem, for me it meant that, for now, it's a no-go.

  • What do you need to get started:
  • Ease of use: if it wasn't for the "we don't like Maven enough to care about it" policy the whole user experience would be a breeze.
  • Documentation: although it looks smaller than Orient's in fact it seems to have everything you need to get started. Also it seems to be well maintained so you shouldn't be confused what's outdated and what's not.
  • Adoption: LinkedIn and... LinkedIn?
  • License: Apache 2.0
  • Protocol: HTTP and bare TCP/IP
  • Misc:
    • linear scalability
    • great latency
  • When to use: with it's replication system and the fact that each node is equal it seems like a great candidate for systems which can be at times under heavy usage by the users and where losing data wouldn't be favourable but when we can wait a bit for eventual consistency to kick-in. How about a shopping cart at a large store?


Conclusions?

For the time being (until Voldemort gets a Maven artifact or we decide to migrate all our data to a NoSQL database in which case Cassandra would be a good fit) I will try to convince my team to use OrientDB:

  • easy to set up. Contrary to Cassandra and Voldemort (lack of maven artifact).
  • distributed architecture with HA out of the box (or at least that's what they say)
  • fast (thanks to the MVRB Tree algorithm)

No comments:

Post a Comment