Here are some of the per­for­mance bench­marks I’ve col­lected per­son­ally, feel free to ref­er­ence them, but do keep in mind that they should be taken at face value only, and as a rule of thumb you should be bench­mark­ing against your actual data and use case.

Dis­claimer : The bench­marks shown here are mostly cre­ated with sim­ple POCOs, and may not be rep­re­sen­ta­tive of your par­tic­u­lar use case.

 

Binary Seri­al­iz­ers

image

image

Ver­sions tested:

 

 

JSON Seri­al­iz­ers

image

image

Ver­sions tested:

 

 

Sorted­Dic­tionary vs Dic­tio­nary vs Map vs Array

image

More info here.

 

 

Cost of throw­ing exceptions

image

image

More info here.

 

 

Hash­Set vs List vs Dictionary

image

image

image

More info here.

20 Responses to “Benchmarks”

  1. Daniel says:

    Hi,

    Both JSON.Net and Ser­viceS­tack has got­ten new releases. Any plans on updating?

    //Daniel

  2. […] and fea­ture rich. SisoDb, on the other hand relies on ServiceStack.Text. Com­pare bench­marks here: http://theburningmonk.com/benchmarks/ These stats com­pares the v4.07 release of JSON.Net and as of now there’s a v4.08 release and […]

  3. Eric Hexter says:

    Could you explain what the mea­sure­ments of the json tests rep­re­sent? What are the mea­sure­ments of the seri­al­iza­tion and dese­ri­al­iza­tion columns? I assume it is the total time taken to per­form the oper­a­tion on a batch num­ber of poco objects? Is that cor­rect ? Thanks, Eric

  4. theburningmonk says:

    Eric — those are aver­age times in mil­lisec­onds, for infor­ma­tion about the test itself, check out: http://theburningmonk.com/2011/08/performance-test-json-serializers/

  5. Omu says:

    hi Yan,
    it’s not clear on the pic­tures which ones are the best, the tall ones or the small ones, you haven’t explained in the text either, although most lazy ppl (me) won’t read the anyway

  6. max says:

    omu,u’d bet­ter use stan­dard serializer,json seri­al­izer and list,if u are that lazy.
    Thanks to Yan,
    Now i won­der how quick is our cus­tom serializer )

  7. […] are some more .NET related per­for­mance bench­marks, also using the ratio com­par­i­son method. This entry was posted in Uncat­e­go­rized by Jack Fox. […]

  8. TheXenocide says:

    Just curi­ous if there are any plans to update these bench­marks? In doing some more sim­ple inde­pen­dent tests right now I’m see­ing JSON.NET out­per­form­ing Ser­viceS­tack (though it’s not per­form­ing as well as I’d like it to and I’ll keep dig­ging to see if there’s a more effi­cient mech­a­nism I can use in either). My object graph is also much more com­plex than those used in these tests, so it’s pos­si­ble it’s just per­for­mance char­ac­ter­is­tics par­tic­u­lar to my use case.

    I’d also be inter­ested in see­ing the per­for­mance of System.Collections.Concurrent added to the col­lec­tion tests so we can com­pare the overhead.

  9. theburningmonk says:

    @Xenocide — yeah, the bench­marks are long over­due for an update (which I’ve been mean­ing to do for a while now), I should hope­fully have some time this week­end to give it a shot!

  10. Mads says:

    Would love to see per­for­mance num­bers for Json.NET 5.0.6, any chance you would get around to update the benchmarks?

    Cheers

  11. theburningmonk says:

    @Mads — sorry for the delay, been mean­ing to update these bench­mark num­bers for a while and just never got around to doing it.

    I have updated both sets of num­bers (for JSON and binary seri­al­iz­ers) and Json.Net 5.0.6 showed pretty big improve­ment on dese­ri­al­iza­tion speed and clos­ing in on ServiceStack’s lead!

  12. Hi, do you have the source code of your bench­mark for the JSON Seri­al­iz­ers somewhere?

  13. Y# says:

    Hi,

    Here’s a new kid on the block, to dese­ri­al­ize JSON either into POCOs or into trees of Dictionaries(string, object) :

    https://github.com/ysharplanguage/JSONParser/blob/master/JsonParser.cs

    It’s merely a parser (dese­ri­al­iza­tion use cases only), but it’s pretty fast and short (less than 450 SLOC total). The pub­lic interface’s Parse method is generic and accepts either a JSON pay­load com­ing in one chunk (string), or via a System.IO.Stream.

    I’ve included some tests and their sam­ple data (from kilo­bytes to megabytes) at the above GitHub.

    Enjoy :)

  14. Y# says:

    Sorry, new home for this exper­i­men­tal parser/deserializer, with updated tests and fig­ures, com­pared to JSON.NET 5.0 r8’s and Ser­viceS­tack 3.9.59’s :

    https://github.com/ysharplanguage/FastJsonParser

  15. Y# says:

    Re-ran the JSON dese­ri­al­ize tests on my lap­top (w/ code below for this System.Text.Json.JsonParser):

    ~ 17% faster than Ser­viceS­tack 3.9.59, ~ 20% faster than fastJ­SON, ~ 29% faster than JSON.NET 5.0 r8

    (464 ms vs. 556 ms vs. 580 ms vs. 651 ms, resp.)

    // speed test Json­Parser
    DoSpeedTest(“JsonParser”, Seri­al­ize­With­J­son­Net, Dese­ri­al­ize­With­J­son­Parser, CountAverageJsonStringPayload);

    //(lines omit­ted)…

    pri­vate sta­tic List DeserializeWithJsonParser(List json­Strings)
    {
    var objects = jsonStrings.Select(new System.Text.Json.JsonParser().Parse).ToList();
    return objects;
    }

Leave a Reply