Performance Test – BinaryFormatter vs Protobuf-Net

Note: don’t for­get to check out the Bench­marks page to see the lat­est round up of binary and JSON serializers.

When work­ing with the Bina­ry­For­mat­ter class fre­quently, one of the things you notice is that it is really damn inef­fi­cient… both in terms of speed as well as the pay­load (the size of the seri­al­ized byte array).

Google’s Pro­to­col Buffers for­mat is designed to be super fast and min­i­mizes the pay­load by requir­ing a seri­al­iz­able object to define ahead of time the order in which its properties/fields should be serialized/deserialized in, doing so removes the need for all sorts of meta­data that tra­di­tion­ally need to be encoded along with the actual data.

Marc Grav­ell (of Stack­Over­flow fame!) has a .Net imple­men­ta­tion called ‘protobuf-net’, and is said to totally kick ass! As with most per­for­mance related top­ics, it’s hugely intrigu­ing to me so I decided to put it to test myself :-)

Assumptions/Conditions of tests

  1. code is com­piled in release mode, with opti­miza­tion option turned on
  2. 5 runs of the same test is per­formed, with the top and bot­tom results excluded, the remain­ing three results is then averaged
  3. 100,000 iden­ti­cal instances of type SimpleObject(see below) is seri­al­ized and deserialized
  4. serialization/deserializatoin of the objects hap­pen sequen­tially in a loop (no concurrency)



Unsur­pris­ingly, the protobuf-net seri­al­izer wins hands-down in all three cat­e­gories, as you can see from the table below it is a stag­ger­ing 12x faster than Bina­ry­For­mat­ter when it comes to seri­al­iza­tion, with a pay­load size less than 15th of its counterpart’s.

One curi­ous obser­va­tion about the pay­load size is that, when I used a Bina­ry­Writer to sim­ply write every prop­erty into the out­put stream with­out any meta­data, what I got back should be the min­i­mum pay­load size with­out com­pres­sion, and yet the protobuf-net seri­al­izer still man­ages to beat that!




Bina­ry­For­mat­ter with ISerializable

I also tested the Bina­ry­For­mat­ter with a class that imple­ments the ISe­ri­al­iz­able inter­face (see below) because oth­ers had sug­gested in the past that you are likely to get a notice­able per­for­mance boost if you imple­ment the ISe­ri­al­iz­able inter­face your­self. The belief is that it will per­form much bet­ter as it removes the reliance on reflec­tion which can be detri­men­tal to the per­for­mance of your code when used excessively.

How­ever, based on the tests I have done, this does not seem to be the case, the slightly bet­ter seri­al­iza­tion speed is far from con­clu­sive and is off­set by a slightly slower dese­ri­al­iza­tion speed..


Source code

If you’re inter­ested in run­ning the tests your­self, you can find the source code here, it uses my Sim­ple­SpeedTester frame­work to orches­trate the test runs but you should be able to get the gist of it fairly easily.

UPDATE 2011/08/24:

As I men­tioned in the post, protobuf-net man­aged to pro­duce a smaller pay­load than what is required to hold all the prop­erty val­ues of the test object with­out any meta.

I posted this ques­tion on SO, and as Marc said in his answer the smaller pay­load is achieved through the use of varint and zigzag encod­ing, read more about them here.