Here are some of the per­for­mance bench­marks I’ve col­lected per­son­ally, feel free to ref­er­ence them, but do keep in mind that they should be taken at face value only, and as a rule of thumb you should be bench­mark­ing against your actual data and use case.

Dis­claimer : The bench­marks shown here are mostly cre­ated with sim­ple POCOs, and may not be rep­re­sen­ta­tive of your par­tic­u­lar use case.

 

Binary Seri­al­iz­ers

image[17]

image[14]

 

JSON Seri­al­iz­ers

image[6]

image[3]

 

Sorted­Dic­tionary vs Dic­tio­nary vs Map vs Array

image

More info here.

 

 

Cost of throw­ing exceptions

image

image

More info here.

 

 

Hash­Set vs List vs Dictionary

image

image

image

More info here.

  • Daniel

    Hi,

    Both JSON.Net and Ser­viceS­tack has got­ten new releases. Any plans on updating?

    //Daniel

  • Pingback: Ranting is good for you « Daniel Wertheim

  • http://hex.lostechies.com Eric Hex­ter

    Could you explain what the mea­sure­ments of the json tests rep­re­sent? What are the mea­sure­ments of the seri­al­iza­tion and dese­ri­al­iza­tion columns? I assume it is the total time taken to per­form the oper­a­tion on a batch num­ber of poco objects? Is that cor­rect ? Thanks, Eric

  • the­burn­ing­monk

    Eric — those are aver­age times in mil­lisec­onds, for infor­ma­tion about the test itself, check out: http://theburningmonk.com/2011/08/performance-test-json-serializers/

  • Omu

    hi Yan,
    it’s not clear on the pic­tures which ones are the best, the tall ones or the small ones, you haven’t explained in the text either, although most lazy ppl (me) won’t read the anyway

  • max

    omu,u’d bet­ter use stan­dard serializer,json seri­al­izer and list,if u are that lazy.
    Thanks to Yan,
    Now i won­der how quick is our cus­tom serializer )

  • Pingback: Performance Test – Json Serializers Part III | theburningmonk.com

  • Pingback: Benchmarking F# Data Structures — Loading Data | craftyThoughts

  • Pingback: Simple Speed Tester – moved to Github! | theburningmonk.com

  • Pingback: Making Amazon SimpleWorkflow simpler to work with | theburningmonk.com

  • Pingback: F# – XmlSerializer, Record types and [CLIMutable] | theburningmonk.com

  • TheXeno­cide

    Just curi­ous if there are any plans to update these bench­marks? In doing some more sim­ple inde­pen­dent tests right now I’m see­ing JSON.NET out­per­form­ing Ser­viceS­tack (though it’s not per­form­ing as well as I’d like it to and I’ll keep dig­ging to see if there’s a more effi­cient mech­a­nism I can use in either). My object graph is also much more com­plex than those used in these tests, so it’s pos­si­ble it’s just per­for­mance char­ac­ter­is­tics par­tic­u­lar to my use case.

    I’d also be inter­ested in see­ing the per­for­mance of System.Collections.Concurrent added to the col­lec­tion tests so we can com­pare the overhead.

  • the­burn­ing­monk

    @Xenocide — yeah, the bench­marks are long over­due for an update (which I’ve been mean­ing to do for a while now), I should hope­fully have some time this week­end to give it a shot!

  • Mads

    Would love to see per­for­mance num­bers for Json.NET 5.0.6, any chance you would get around to update the benchmarks?

    Cheers

  • the­burn­ing­monk

    @Mads — sorry for the delay, been mean­ing to update these bench­mark num­bers for a while and just never got around to doing it.

    I have updated both sets of num­bers (for JSON and binary seri­al­iz­ers) and Json.Net 5.0.6 showed pretty big improve­ment on dese­ri­al­iza­tion speed and clos­ing in on ServiceStack’s lead!

  • Pingback: Binary and JSON serializer benchmarks updated | theburningmonk.com

  • http://blog.codebeside.org Gus­tavo Guerra

    Hi, do you have the source code of your bench­mark for the JSON Seri­al­iz­ers somewhere?

  • https://github.com/ysharplanguage/JSONParser Y#

    Hi,

    Here’s a new kid on the block, to dese­ri­al­ize JSON either into POCOs or into trees of Dictionaries(string, object) :

    https://github.com/ysharplanguage/JSONParser/blob/master/JsonParser.cs

    It’s merely a parser (dese­ri­al­iza­tion use cases only), but it’s pretty fast and short (less than 450 SLOC total). The pub­lic interface’s Parse method is generic and accepts either a JSON pay­load com­ing in one chunk (string), or via a System.IO.Stream.

    I’ve included some tests and their sam­ple data (from kilo­bytes to megabytes) at the above GitHub.

    Enjoy :)

  • https://github.com/ysharplanguage/JSONParser Y#

    Sorry, new home for this exper­i­men­tal parser/deserializer, with updated tests and fig­ures, com­pared to JSON.NET 5.0 r8’s and Ser­viceS­tack 3.9.59’s :

    https://github.com/ysharplanguage/FastJsonParser

  • https://github.com/ysharplanguage/FastJsonParser Y#

    Re-ran the JSON dese­ri­al­ize tests on my lap­top (w/ code below for this System.Text.Json.JsonParser):

    ~ 17% faster than Ser­viceS­tack 3.9.59, ~ 20% faster than fastJ­SON, ~ 29% faster than JSON.NET 5.0 r8

    (464 ms vs. 556 ms vs. 580 ms vs. 651 ms, resp.)

    // speed test Json­Parser
    DoSpeedTest(“JsonParser”, Seri­al­ize­With­J­son­Net, Dese­ri­al­ize­With­J­son­Parser, CountAverageJsonStringPayload);

    //(lines omit­ted)…

    pri­vate sta­tic List DeserializeWithJsonParser(List json­Strings)
    {
    var objects = jsonStrings.Select(new System.Text.Json.JsonParser().Parse).ToList();
    return objects;
    }

  • http://omidid.github.io/FastJsonWriter/ Omid

    Hi, try my project to seri­al­ize JSON.
    http://omidid.github.io/FastJsonWriter/

  • rpg­maker

    Hi, Can you add my project for seri­al­iz­ing to JSON.
    https://github.com/rpgmaker/NetJSON

  • Yan Cui

    Sure, is it avail­able on Nuget? Would be help­ful if you can give me some info (or add them to the repo) on how to use it and if there’s any con­sid­er­a­tions I should be aware of to get the most per­for­mance from your serializer?

  • rpg­maker

    It is not cur­rently avail­able on Nuget, But I can get it on there by end of today if it is makes it eas­ier for you. There is noth­ing really that I can come up with in term of get­ting the most per­for­mance. The cur­rent inter­face takes just an object and return string and vice versa. I will try to add more infor­ma­tion to the site about the Usage.

  • rpg­maker

    In addi­tion my pre­vi­ous com­ment. You can quickly get to the gen­er­ated dll for the project by nav­i­gat­ing to this uri: https://github.com/rpgmaker/NetJSON/tree/master/NetJSON/lib/net40

  • Yan Cui

    If you can make it avail­able on Nuget that’d make life eas­ier for me to main­tain the test suite going for­ward. Drop me a line when that’s avail­able and I’ll add that to the list.

  • rpg­maker

    Sorry for spam­ming your blog, I just went ahead and uploaded it to nuget:

    https://www.nuget.org/packages/NetJSON/

  • rpg­maker
  • Yan Cui

    JSON bench­marks has been updated to include NetJ­son now :-)

  • Cocoa

    I tested this :: https://www.nuget.org/packages/NetJSON/, and com­pared it with pro­to­buf, your com­par­isons are way full of it.

  • Cocoa

    Are you pur­posely try­ing to impose peo­ple use less effi­cient libraries???

  • Yan Cui

    feel free to pub­lish your own results, when I tested it there was no dis­tin­guish­able dif­fer­ence between NetJ­son and Proto-buf in terms of speed

  • Yan Cui

    I don’t know what you mean by that, I merely pub­lish the results of tests that I run

  • Y#

    Just a sug­ges­tion, as I’ve recently added JSON­Path sup­port to System.Text.Json:

    how about includ­ing a few JSON­Path selec­tion speed tests across the var­i­ous imple­men­ta­tions, with tim­ings that would either include or exclude the prior over­heads of the pars­ing times? Just an idea.

    Out of mere curios­ity, I’ve done it if only in one unit test with the implicit goal of keep­ing the applica­tive code as straight­for­ward / naive as pos­si­ble in regards to the respec­tive JSON dese­ri­al­izer APIs com­bined with JSON­Path, and more specif­i­cally, in the case of Json.NET vs. System.Text.Json.

    Vis­i­ble here: https://github.com/ysharplanguage/FastJsonParser/blob/master/JsonTest/ParserTests.cs#L1005

    With some basic doc­u­men­ta­tion there: https://github.com/ysharplanguage/FastJsonParser#JSONPath

    HTH,

  • Y#

    Slight per­for­mance improve­ment in ver­sion 2.0.0.3, along with sup­port for dese­ri­al­iza­tion into anony­mous types and nul­lable types, bet­ter inte­gral types sup­port, and sup­port for System.Guid (prior missing):

    https://www.nuget.org/packages/System.Text.Json

    HTH,