MS Bond and Chiron benchmarked

DISCLAIMER : as always, you should bench­mark against your pay­load and use case, the bench­mark num­bers I have pro­duced here is unlike­ly to be rep­re­sen­ta­tive of your use cas­es and nei­ther is any­body else’s bench­mark num­bers.

You can use the sim­ple test har­ness I cre­ated and see these exam­ple code to bench­mark against your par­tic­u­lar pay­load.

 

I updat­ed my bina­ry and JSON seri­al­iz­ers bench­mark ear­li­er this week, and got some feed­backs on new seri­al­iz­ers that I have missed, name­ly Chi­ron and Microsoft’s Bond. Here, we’ll have a look how the two fared in the bench­mark.

 

MS Bond

Microsoft announced their answer to Google’s Pro­to­col Buffer with Bond this time last year (Jan 2015). Final­ly I’ve got around to actu­al­ly test it out (after an ex-Gamesys col­league com­ment­ed on the last update — thanks Rob!).

First, you define you con­tract with a .bond file (see tuto­r­i­al here), for exam­ple…

bond_benchmark_03

Now you run the Bond com­pil­er tool, gbc, against this file to gen­er­ate a C# class that looks like this…

bond_benchmark_04

To seri­al­ize and dese­ri­al­ize data, you also need to add the Bond C# nuget pack­age to your project and fol­low the exam­ples in the afore­men­tioned tuto­r­i­al.

Here’s how Bond fared against oth­er bina­ry seri­al­iz­ers on my list.

NOTE: there’s an updat­ed bench­mark test that uses a dif­fer­ent ini­tial buffer size which makes a huge dif­fer­ence in per­for­mance for Bond. Please read the linked post for more info.

bond_benchmark_01

bond_benchmark_02

The result makes for an inter­est­ing read­ing…

  • Bond pro­duced the small­est pay­load, and is the fastest at dese­ri­al­iz­ing the pay­load by some dis­tance.
  • It is also the slow­est at seri­al­iz­ing the pay­load!

 

Chiron

I read about Chi­ron in Mar­cus Griep’s F# advent post but then for­got about it (total­ly my bad… too many hours on Blood­borne over xmas, such an awe­some game ).

Any­ways, Chi­ron has a F#-friendly API but because it uses sta­t­i­cal­ly resolved type para­me­ters you can’t use it from C#.

In order to serialize/deserialize a type, the type needs to define the sta­t­ic meth­ods ToJ­son and FromJ­son. The inlined seri­al­ize and dese­ri­al­ize func­tions can then con­straint your type to have those sta­t­ic mem­bers and invoke them in the cor­re­spond­ing func­tion. I used the same tech­nique in MBrace.AWS and hon­est­ly, I’m not hap­py with the amount of work this push­es onto the user, espe­cial­ly when they end up hav­ing to write unin­ter­est­ing plumb­ing code…

On the API front, I’m not thrilled with the cus­tom oper­a­tors either, even though there are only 3 of them so I’m prob­a­bly just over-react­ing. In gen­er­al I find cus­tom oper­a­tors get in the way of dis­cov­ery.

Read­ing through the post, this para­graph sug­gests a lot of inter­me­di­ate JsonResult<‘a> and Json objects are cre­at­ed dur­ing the seri­al­iza­tion process. Whilst this might be an idiomat­ic func­tion­al approach, it’s also like­ly to hurt our per­for­mance..

The *> oper­a­tor that we used in ToJson dis­cards the JsonResult<'a> (which is only used when writ­ing), but con­tin­ues to build upon the Json object from the pre­vi­ous oper­a­tion. By chain­ing these oper­a­tions togeth­er, we build up the mem­bers of a Json.Object.

Unsur­pris­ing­ly, the cost of immutabil­i­ty proved real­ly cost­ly under the bench­mark.

chiron_benchmark_01

chiron_benchmark_02

 

So that’s it folks, anoth­er 2 seri­al­iz­ers added to our sta­ble. If there any oth­er seri­al­iz­ers that you think I should include here, please give me a shout and I’ll do my best to accom­mo­date.