Performance Test – BinaryFormatter vs Protobuf-Net

Note: don’t for­get to check out the Bench­marks page to see the lat­est round up of bina­ry and JSON seri­al­iz­ers.

When work­ing with the Bina­ry­For­mat­ter class fre­quent­ly, one of the things you notice is that it is real­ly damn inef­fi­cient… both in terms of speed as well as the pay­load (the size of the seri­al­ized byte array).

Google’s Pro­to­col Buffers for­mat is designed to be super fast and min­i­mizes the pay­load by requir­ing a seri­al­iz­able object to define ahead of time the order in which its properties/fields should be serialized/deserialized in, doing so removes the need for all sorts of meta­da­ta that tra­di­tion­al­ly need to be encod­ed along with the actu­al data.

Marc Grav­ell (of Stack­Over­flow fame!) has a .Net imple­men­ta­tion called ‘pro­to­buf-net’, and is said to total­ly kick ass! As with most per­for­mance relat­ed top­ics, it’s huge­ly intrigu­ing to me so I decid­ed to put it to test myself :-)

Assumptions/Conditions of tests

  1. code is com­piled in release mode, with opti­miza­tion option turned on
  2. 5 runs of the same test is per­formed, with the top and bot­tom results exclud­ed, the remain­ing three results is then aver­aged
  3. 100,000 iden­ti­cal instances of type SimpleObject(see below) is seri­al­ized and dese­ri­al­ized
  4. serialization/deserializatoin of the objects hap­pen sequen­tial­ly in a loop (no con­cur­ren­cy)

image

Results

Unsur­pris­ing­ly, the pro­to­buf-net seri­al­iz­er wins hands-down in all three cat­e­gories, as you can see from the table below it is a stag­ger­ing 12x faster than Bina­ry­For­mat­ter when it comes to seri­al­iza­tion, with a pay­load size less than 15th of its counterpart’s.

One curi­ous obser­va­tion about the pay­load size is that, when I used a Bina­ry­Writer to sim­ply write every prop­er­ty into the out­put stream with­out any meta­da­ta, what I got back should be the min­i­mum pay­load size with­out com­pres­sion, and yet the pro­to­buf-net seri­al­iz­er still man­ages to beat that!

image

image

image

Bina­ry­For­mat­ter with ISe­ri­al­iz­able

I also test­ed the Bina­ry­For­mat­ter with a class that imple­ments the ISe­ri­al­iz­able inter­face (see below) because oth­ers had sug­gest­ed in the past that you are like­ly to get a notice­able per­for­mance boost if you imple­ment the ISe­ri­al­iz­able inter­face your­self. The belief is that it will per­form much bet­ter as it removes the reliance on reflec­tion which can be detri­men­tal to the per­for­mance of your code when used exces­sive­ly.

How­ev­er, based on the tests I have done, this does not seem to be the case, the slight­ly bet­ter seri­al­iza­tion speed is far from con­clu­sive and is off­set by a slight­ly slow­er dese­ri­al­iza­tion speed..

image

Source code

If you’re inter­est­ed in run­ning the tests your­self, you can find the source code here, it uses my Sim­ple­SpeedTester frame­work to orches­trate the test runs but you should be able to get the gist of it fair­ly eas­i­ly.

UPDATE 2011/08/24:

As I men­tioned in the post, pro­to­buf-net man­aged to pro­duce a small­er pay­load than what is required to hold all the prop­er­ty val­ues of the test object with­out any meta.

I post­ed this ques­tion on SO, and as Marc said in his answer the small­er pay­load is achieved through the use of varint and zigzag encod­ing, read more about them here.