MessagePack: Efficient, Cross Language Binary Object Serialization
MessagePack GitHub repo is a new binary-based object serialization protocol and library built with efficiency and speed in mind. Developer Sadayuki Furuhashi presents it as a faster alternative to JSON that has similarly broad support across several popular languages.
Serialization is the process of taking an object (such as a string, hash, or even something of your own creation) and turning it into a single time of data that can be transmitted down a network connection, stored in a file, or similar. Protocols like JSON, YAML, and XML are commonly used for this purpose, but for the highest efficiency, binary protocols are an alternative. One option available to all Ruby developers (as it's in the standard library) is the Marshal library but this protocol isn't cross-language friendly (or even between some Ruby versions).
To demonstrate MessagePack's speed, the developer presents the results of a benchmark where 200,000 objects made up of three integers and a 512 byte string were serialized and deserialized once each:
While Ruby is the focus, there are also libraries for Python, Perl, C/C++, Haskell, and Lua ready to roll.
Installation and Example
To install MessagePack:
gem install msgpackExample:
require 'msgpack' msg = [1,2,3].to_msgpack #=> "x93x01x02x03" MessagePack.unpack(msg) #=> [1,2,3]
March 20, 2010 at 4:31 pm
MessagePack.pack({ :lol => :wut })
NoMethodError: undefined method `to_msgpack' for :lol:Symbol
holy crap
March 20, 2010 at 7:49 pm
This sounds a lot like BSON (http://bsonspec.org). I would be interested in seeing a comparison if that in fact was the case.
March 20, 2010 at 9:37 pm
The big win for JSON is how easily in-browser clients can consume the payload. This does look interesting for scenarios when neither serialization endpoint is a browser (or if there were a lightweight JavaScript implementation).
March 20, 2010 at 10:05 pm
sounds good for Interprocess Communication or talking between basic TCP/IP based apps but for web stuff I'd definitely stick with JSON... seems kind of stupid to even have JSON on that graph to be honest.
March 21, 2010 at 10:43 am
>> (10**100).to_msgpackRangeError: bignum too big to convert into `unsigned long long'
from (irb):8:in `to_msgpack'
from (irb):8
This can be a problem if you expect all your ruby numbers to be packable.
March 21, 2010 at 2:40 pm
BERT is another binary protocol that looks very similar on the surface and supports a bunch of languages as well: http://bert-rpc.org/
March 22, 2010 at 3:50 pm
A while back I did some benchmarking vs. JSON (C), Marshal, YAJL, BSON (C) and Bert. MessagePack comes out on top. http://gist.github.com/290425
March 22, 2010 at 7:54 pm
And as many have already noted, type support for MessagePack seems to be lacking (e.g. no symbols, no bignums)
BERT seems to have these problems handled a little better