NOTE: For an updated set of bench­marks, see the Bench­marks page.

For those of you who have worked with JSON data in .Net you should be famil­iar with the Dat­a­Con­trac­tJ­son­Se­ri­al­izer class and the JavaScript­Se­ri­al­izer class already. Both allow you to serialize/deserialize an object to and from a piece of string in JSON for­mat, though there are some notable dif­fer­ences.

Besides these two BCL (base class library) JSON seri­al­iz­ers, there are pop­u­lar third-party offer­ing such as Json.Net and a rel­a­tively newer ServiceStack.Text which also offers its own seri­al­iza­tion for­mat called JSV (JSONCSV).

It is claimed that ServiceStack.Text’s JSON seri­al­izer is 3x faster than Json.Net and 3.6x faster than the BCL JSON seri­al­iz­ers! So, nat­u­rally, I had to test it out for myself and here’s what I found.

Assumptions/Conditions of tests

  1. code is com­piled in release mode, with opti­miza­tion options turned on
  2. 5 runs of the same test is per­formed, with the top and bot­tom results excluded, the remain­ing three results is then averaged
  3. 100,000 instances of type Sim­pleOb­ject (see below) is cre­ated, each with a dif­fer­ent ID and Name, and then given to the seri­al­iz­ers to seri­al­ize and deserialize
  4. serialization/deserialization of the objects hap­pen sequen­tially in a loop (no concurrency)
 1: [DataContract]
 2: public class SimpleObject
 3: {
 4:     [DataMember]
 5:     public int Id { get; set; }
 6:
 7:     [DataMember]
 8:     public string Name { get; set; }
 9: }

Results

With these assump­tions in mind, here are the aver­age times (in mil­lisec­onds) I recorded for seri­al­iza­tion and dese­ri­al­iza­tion using each of five JSON seri­al­iz­ers I tested:

image

image

Look­ing at these data, it would seem the ServiceStack.Text.JsonSerializer class offers the best speed in both seri­al­iza­tion and dese­ri­al­iza­tion cases, how­ever the gains are much more mod­est than those adver­tised in the case of seri­al­iza­tion. How­ever, it really comes into its own when it comes to dese­ri­al­iza­tion and the speed gains are quite impres­sive indeed!

Update 2011/09/12:

Turns out I had a typo in my per­for­mance test code and I was using Json.Net for the seri­al­iza­tion test for Sim­ple­J­son, my bad, sorry folks… So I fixed the typo and ran the tests again and updated the data and graph above with the cor­rect data. As Pra­bir Shrestha pointed out, by enabled Reflection.Emit you’re able to get much bet­ter per­for­mance out of Sim­ple­J­son and the new test results reflect this as the Sim­ple­J­son test was run whilst Reflection.Emit was enabled.

Share

17 Responses to “Performance Test — JSON serializers”

  1. Demis Bellot says:

    Hey cool nice benchmarks :)

    Note the ini­tial speed gains we’re based on the North­wind DataSet bench­marks (also linked to on the ref arti­cle at: http://goo.gl/AhHUp ) which shows the aver­age per­for­mance after com­bin­ing both Seri­al­iza­tion and Dese­ri­al­iza­tion times for each table in the North­wind data­base. Which are sim­i­lar to your bench­marks where most of the speed gains came from dese­ri­al­iza­tion. The rea­son for this is sim­ple, it’s hard to make writ­ing to an out­put stream any quicker and most seri­al­iz­ers will gen­er­ally do it the same naive way.

    So for com­par­isons your bench­marks (after com­bin­ing Seri­al­iza­tion and Dese­ri­al­iza­tion) show ServiceStack.Text is 3.697x faster than JSON.NET

  2. Livingston says:

    Could you include Sim­ple­J­SON in your com­par­isons?
    http://simplejson.codeplex.com

  3. […] var addthis_config = {“data_track_clickback”:true};The Burn­ing Monk blog has a nice per­for­mance post about how dif­fer­ent .NET JSON libraries per­form. The win­ner? ServiceStack.Text. I per­son­ally use […]

  4. theburningmonk says:

    @Livingston — sure, I can add that to the post later on, if you’re inter­ested though, the source code for the test is included in the exam­ples project for my Sim­ple­SpeedTester project:

    http://simplespeedtester.codeplex.com

    You should be able to put together a test your­self eas­ily enough, I would appre­ci­ate any feed­back on the tester too :-)

  5. theburningmonk says:

    @Demis Bel­lot — that’s a very com­pre­hen­sive set of bench­marks you’ve got there! Love the work you guys have done on the JSON seri­al­izer and Red­dis client btw :-)

  6. Demis Bellot says:

    @theburningmonk hey no wor­ries thanks, glad it’s prov­ing useful :)

  7. theburningmonk says:

    @Livingston — Sim­ple­J­son included in the test as requested ;-)

  8. Livingston says:

    oh wow.… I expected bet­ter per­for­mance than that. Thanks for adding though. I’ll check out the tester later today

  9. Thanks for adding Sim­ple­J­son to your benchmarks.

    i down­loaded the source code and found out that you are actu­ally using json.net for seri­al­iz­ing tests in simplejson.

    // speed test Sim­ple­J­son
    DoSpeedTest(“SimpleJson”, Seri­al­ize­With­J­son­Net, DeserializeWithSimpleJson);

    Seri­al­ize­With­J­son­Net should be Seri­al­ize­With­Sim­ple­J­son instead.

    And one more thing. by default lot of fea­tures are dis­abled as we tend to sup­port .net 2.0+, sl3+ and wp7 using the same code. Since sl and wp7 doesnt con­tain Reflection.Emit we have dis­abled it by default. to enable it you must man­u­ally define the SIMPLE_JSON_REFLECTIONEMIT as part of con­di­tional com­pi­la­tion sym­bol or uncom­ment line num­ber 18.
    #define SIMPLE_JSON_REFLECTIONEMIT

    I had also writ­ten a bench­mark app before ship­ping sim­ple json pub­lic and it was very close to json.net. there are times when it actu­ally per­forms bet­ter than json.net

    Here is the result i got after enabling Reflection.Emit for sim­ple json

    Test Group [Json.Net], Test [Seri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [964.4081] mil­lisec­onds
    Test Group [Json.Net], Test [Dese­ri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [4175.15426666667] mil­lisec­onds
    Test Group [ServiceStack.Text], Test [Seri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [597.442066666667] mil­lisec­onds
    Test Group [ServiceStack.Text], Test [Dese­ri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [670.168766666667] mil­lisec­onds
    Test Group [Dat­a­Con­trac­tJ­son­Se­ri­al­izer], Test [Seri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [1152.636] mil­lisec­onds
    Test Group [Dat­a­Con­trac­tJ­son­Se­ri­al­izer], Test [Dese­ri­al­iza­tion] results summary:

    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [3614.29796666667] mil­lisec­onds
    Test Group [JavaScript­Se­ri­al­izer], Test [Seri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [2517.56013333333] mil­lisec­onds
    Test Group [JavaScript­Se­ri­al­izer], Test [Dese­ri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [6903.22913333333] mil­lisec­onds
    Test Group [Sim­ple­J­son], Test [Seri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [953.122566666667] mil­lisec­onds
    Test Group [Sim­ple­J­son], Test [Dese­ri­al­iza­tion] results sum­mary:
    Suc­cesses [5]
    Fail­ures [0]
    Aver­age Exec Time [3210.8124] mil­lisec­onds
    all done…

  10. […] on from my pre­vi­ous test which showed that the ServiceStack.Text JSON seri­al­izer was the top dog in town, I came across […]

  11. […] to ServiceStack.Text for our JSON seri­al­iza­tion duties a while ago because it was blind­ingly fast, much faster than any other JSON seri­al­izer we could find for .NET. I think you’ll agree that all of Demis’ open source work is of sim­i­larly high […]

  12. Another great library worth adding to the bench­mark is fastJson

  13. theburningmonk says:

    @Anastasiosyal — there’s a more up-to-date post which includes fastJ­son, check out the bench­marks page : http://theburningmonk.com/benchmarks/

  14. rrjp says:

    I won­der how JsonFx would stack up?

  15. rrjp says:

    Ah, nev­er­mind. Thanks for the code! I added JsonFx and ran it. This chart will prob­a­bly look bad, but…

    Name Seri­al­ize Dese­ri­al­ize
    ServiceStack.Text 410.0747667 624.4737
    fastJ­son 814.4741667 823.0015333
    Json.Net 919.5261667 1350.576233
    Sim­ple­J­son 949.6640333 2334.040933
    Dat­a­Con­trac­tJ­son­Se­ri­al­izer 772.007 2351.076167
    JsonFx 1293.9739 2359.188467
    JavaScript­Se­ri­al­izer 2082.950367 4094.520433
    Jay­Rock 4458.4778 5638.371767

  16. theburningmonk says:

    @rrjp — I’ve included JsonFx in the Json Seri­al­iz­ers bench­mark in the bench­marks page

  17. […] I started doing these bench­marks more than two years ago, I have been impressed by the way Json.Net upped its game since those early bench­marks! Long may […]

Leave a Reply