JSON serializers benchmarks updated

DISCLAIMER : as always, you should bench­mark against your pay­load and use case, the bench­mark num­bers I have pro­duced here is unlikely to be rep­re­sen­ta­tive of your use cases and nei­ther is any­body else’s bench­mark numbers.

You can use the sim­ple test har­ness I cre­ated and see these exam­ple code to bench­mark against your par­tic­u­lar payload.

 

Json.Net, ServiceStack.Text, MongoDB Drive and Jil were all updated to the latest version.

RpgMaker’s NetJson serializer has also been added to the mix and the results are really impressive with a level of performance that’s almost identical to protobuf-net!

image

image

 

Versions tested:

Jil 1.7.0
ServiceStack.Text 4.0.24
Json.Net 6.0.4
fastJson 2.1.1.0
MongoDB Drive 1.9.2
System.Json 4.0.20126.16343
System.Text.Json 1.9.9.1
JsonFx 2.0.1209.2802
JayRock 0.9.16530
  • rpgmaker

    Thanks for the benchmark. By including the min and max for the outcome, NetJSON beats Protobuf-net all the time. Can you post a result that does not eliminate the min and max of the result so as to to see what different it makes?

    Thanks again.

  • rpgmaker

    Here is my result. I excluded the other serializers to eliminate noises and left just netjson and protobuf-net.

    – With Outcome Filter of Min and Max

    Test Group [Protobuf-Net], Test [Serialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [144.405366666667] milliseconds

    Test Group [Protobuf-Net] average serialized byte array size is [51.7227]

    Test Group [Protobuf-Net], Test [Deserialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [236.163266666667] milliseconds

    ———————————————————

    Test Group [NetJson], Test [Serialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [151.2176] milliseconds

    Test Group [NetJson] average serialized byte array size is [98.86646]

    Test Group [NetJson], Test [Deserialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [252.5274] milliseconds

    ———————————————————

    – Without Outcome Filter of Min and Max

    Test Group [Protobuf-Net], Test [Serialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [257.9771] milliseconds

    Test Group [Protobuf-Net] average serialized byte array size is [51.72424]

    Test Group [Protobuf-Net], Test [Deserialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [251.4977] milliseconds

    ———————————————————

    Test Group [NetJson], Test [Serialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [157.98844] milliseconds

    Test Group [NetJson] average serialized byte array size is [98.86456]

    Test Group [NetJson], Test [Deserialization] results summary:

    Successes [5]

    Failures [0]

    Average Exec Time [252.18208] milliseconds

    ———————————————————

  • Pingback: F# Weekly #32, 2014 | Sergey Tihon's Blog()

  • Yan Cui

    A lot of serializers incur a ‘start-up’ cost (reflection, etc. before these are cached) and that cost is amortized over time which is why they’re excluded from the results.

  • rpgmaker

    I guess that make sense. So it seem like protobuf-net would be one of the serializers that incur a start up cost due to all the reflection and caching it does.

  • ??????? ???????????

    Why not just contibute to Json.NET?
    Json.NET is used widely, even in ASP.NET MVC4, it’s unpossible to switch it to NetJson BUT it’s easy to update Json.NET binary assembly with updated version.

  • Cocoa

    NetJSON = fastest out there, besides protobuf isn’t even JSON, and it still is barely faster, that is also excluding the startup costs of Protobuf.. poor protobuf;

  • Yan Cui

    I started to include protobuf in there as a guide so that everyone could see that there’s a difference in terms of performance between binary and BSON serializers so that they can make more informed decision as to which to use (http://theburningmonk.com/2012/08/performance-test-json-serializers-part-iii/).

    The fact that NetJson blurs that line is amazing news, and without putting them side-by-side how would one know that it does?

  • rpgmaker

    When you do get a chance, can you rerun the benchmark again with the latest version of NetJSON 1.0.2? Based on your simpleTester it beats protobuf-net no matter if you include or exclude the min/max time.

    Thanks,

  • Paul Westcott

    NetJson doesn’t appear to escape strings, so probably shouldn’t be included as a json serialiser – i.e. it’s use is severly limited.

  • rpgmaker

    The logic to escape strings is already in the code. It is just not been called because I want to test it well before enabling it. So far adding it does not make it slower than protobuf-net. It is still faster than all the json serializer on the list.

    I have an issue open for since the beginning: https://github.com/rpgmaker/NetJSON/issues/1

  • rpgmaker
  • Ivan Artemov

    I don’t understand, why all JSON serializers use PascalCase notation by default ? In generally, we use JSON for UI and JavaScript is CamelCase object notation.
    fastJson and NetJson can’t use CamelCase as base functionality (not possible to use options)
    NetJSON throw Exceptions on very simple data and not useful.

  • Ivan Artemov

    NetJSON totally unusable. Exceptions on simple data

  • Tj

    You can set case sensitive flag as an option if you are worried about specific naming convention that is not part of json spec in term naming a property without the correct casing

  • Tj

    It will nice if you are bit specific on what type of simple data it is failing on. If you are referring to using anonymous type then you are correct. But in regards to complicated data structures I have yet to see any issues with it

  • Tj

    To better what is currently support please refer to https://github.com/rpgmaker/NetJSON/wiki/Quick-Guide

  • Michael Angel

    In my testing NetJSON did not handle the following:
    1 – Classes that contain collections of Interfaces or abstract class (e.g.List)
    2 – Classes that implement IEnumerable<DictionaryEntry>
    3 – Classes that have several equals operator overloads
    4 – Classes that implement an interface with generic parameters.