WCF – Improve performance with greater concurrency

As good and innovative as WCF is, it also introduced a lot of new complexities and whilst it is easy to get something up and running quickly it takes much more understanding to make your service perform as well as it could.

There are many things you need to consider such as binding types, serialization type, Datatable or POCO, etc. etc. and any of these choices can have a telling effect on the overall performance of your WCF service. Scott Weinstein wrote a very good article on how to create a high performance WCF service (see reference section) using 6 simple steps.

Without going into the subjects of binding selection and data normalization I want to just focus on how you can achieve greater concurrency because for services hosted on the web you won’t be able to use NetTcpBinding and data normalization is almost irrelevant because you won’t (or at least shouldn’t!) be sending large amounts of data back and forth.

Increase the throttling limits

Generally speaking, the most common way to improve the performance of a WCF service is to encourage greater concurrency and if you have used WCF before then chances are you’ve had to change the default throttling behaviour configuration because the defaults are too low for any real world applications to be useful.

These defaults are set to ensure your service is safe from DOS attacks but unfortunately also means your service will run in lock-down mode by default. They have since been raised to more sensible numbers in the new WCF 4 release:

MaxConcurrentSessions MaxConcurrentCalls MaxConcurrentInstances
WCF 3.5 SP1 10 16 26
WCF 4 100 * Processor Count 16 * Processor Count 116 * Processor Count

The new defaults in WCF 4 should provide a good guideline for you when configuring the ServiceThrottlingBehavior of your service (assuming you’re not using WCF 4 already).

Use the PerCall or PerSession instance context mode

The InstanceContextMode also plays a significant role in the overall performance of your service, and of the three options available to you – PerCall, PerSession and Singleton – you should consider PerCall or PerSession for a highly scalable WCF service.

Whilst the PerCall instance context mode is generally regarded as the most scalable option it does carry with it the need to create a instance of your class for each request and you need to ensure that 1) you have a parameterless constructor, and 2) this constructor should do as little as possible. If there are any significant steps that need to be performed, such as loading some reference data, you should avoid doing these in the parameterless constructor so they aren’t performed for each request:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
public class MyService : IMyService
{
    public MyService()
    {
        ReferenceData = LoadReferenceData(); // will be called for EACH request…
    }

    public MyReferenceData ReferenceData { get; private set; }
    …
}

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
public class MyService2 : IMyService
{
    // called the first time the reference data is used and shared across all instances
    private static MyReferenceData ReferenceData = LoadReferenceData();

    public MyService2()
    {
        // ideal constructor which does nothing
    }

    …
}

In cases where the initialization steps are lengthy and unavoidable, or your class require a number of parameters in the constructor (for instance, when you programmatically host a service retrieve from an IoC container) the parameterless constructor can become a problem. To get around this, you could create a wrapper for your class and expose the wrapper as the service instead but hold a static instance of the underlying service which all the requests are passed on to:

[ServiceBehavior(InstanceContextMode = InstanceContextMode.PerCall)]
public class MyServiceWrapper : IMyServiceWrapper
{
    // get the underlying service from the container
    private static IMyService MyService = _container.Resolve<IMyService>();

    public MyServiceWrapper()
    {
        // parameterless constructor which does nothing, so easy to constructor
    }
    public void DoSomething()
    {
        MyService.DoSomething();
    }
}

// dummy interface to ensure the wrapper has the same methods as the underlying service
// but helps to avoid confusion
public interface IMyServiceWrapper : IMyService
{
}

For a sessionful service, the PerSession instance context mode gives you all the benefit of the PerCall instance context mode and at the same time reduces the overhead you pay for that extra concurrency because new instances of your class are no longer created for each request but for each session instead.

If your service is session dependant then you should definitely go with PerSession, but beware, if the channel does not create a session your service will behave as if it was a PerCall service.

Increase the number of idle IO threads in the thread pool

Perhaps the most overlooked aspect when it comes to increasing concurrency of a WCF service. If you’ve set your service to use PerCall or PerSession instance context mode and upped the throttling settings, but are still not getting the response times you’re looking for, then it’s worth investigating whether the calls are being queued because there is not enough IO threads in the ThreadPool to handle the requests.

You can establish whether or not the requests are actually taking longer to process under load (as opposed to being queued at a service level) either by profiling locally or using some form of run-time logging (I wrote a LogExecutionTime attribute which might come in handy). If the calls aren’t taking longer to process and you’re not seeing very high CPU utilisation then the increase in response time is likely a result of the request being queued whilst WCF waits for a new IO thread to be made available to handle the request.

WCF uses the IO threads from the ThreadPool to handle requests and by default, the ThreadPool keeps one IO thread around for each CPU. So on a single core machine that means you only have ONE available IO thread to start with, and when more IO threads are needed they’re created by the ThreadPool with a delay:

“The thread pool maintains a minimum number of idle threads. For worker threads, the default value of this minimum is the number of processors. The GetMinThreads method obtains the minimum numbers of idle worker and I/O completion threads.

When all thread pool threads have been assigned to tasks, the thread pool does not immediately begin creating new idle threads. To avoid unnecessarily allocating stack space for threads, it creates new idle threads at intervals. The interval is currently half a second, although it could change in future versions of the .NET Framework.

If an application is subject to bursts of activity in which large numbers of thread pool tasks are queued, use the SetMinThreads method to increase the minimum number of idle threads. Otherwise, the built-in delay in creating new idle threads could cause a bottleneck.”

However, as WenLong Dong pointed out in his blog (see references section), raising the MinIOThreads setting in the ThreadPool doesn’t work as you’d expect in .Net 3.5 because of a known issue with the ThreadPool which has since been fixed in .Net 4. So if you’re still running .Net 3.5 like most of us, then you will need to go and grab the hotfix from here:

http://support.microsoft.com/kb/976898

Parting thoughts:

Performance tuning isn’t an exact science, and you have to make a case by case judgement on how best to approach your performance issues. Encouraging greater concurrency is just one of the ways you can improve performance, but it’s by no means a silver bullet! In fact, if you go too far down the concurrency route you could find yourself facing a number of problems:

  • 100% CPU – excessive number of concurrent threads can max out your CPU and a lot of CPU time can be wasted on context switching whilst your service becomes less responsive.
  • DOS attack – same as above, but intended by a would be attacker.
  • OutOfMemoryException – if your service is returning a large set of data from database/committing large amount of database writes within a transaction, it’s not unthinkable that you might run into the dreaded OutOfMemoryException given that: 1) in practice you only have a per process cap of around 1.2 ~ 1.5GB of memory and each active thread is allocated a 1MB of memory space (regardless of how much it actually uses); 2) each concurrent call performs large number of object creations which takes up available memory space until they’re garbage collected 3) writing to the Database within a transaction adds to the transaction log which also eats into the available memory.

References:

WenLong Dong’s post on WCF responses being slow and SetMinThreads does not work

WenLong Dong’s post on WCF request throttling and Server Scalability

WenLong Dong’s post on WCF becoming slow after being idle for 15 seconds

Scott Weinstein’s post on creating high performance WCF services

Dan Rigsby’s post on throttling WCF service and maintaining scalability

.Net Tips – Using custom ServiceThrottlingAttribute to specify WCF service throttling behaviour

If you have created a WCF service in the past then I assume you’re aware that WCF is very heavily configuration-driven and that you can specify the service behaviour including the throttling parameters (MaxConcurrentCalls, MaxConcurrentInstances, MaxConcurrentSessions) in the config file.
But to specify the type of service (PerCall, PerSession or Singleton) you need to apply the ServiceBehaviour attribute to your type.
Now, wouldn’t it be nice if you can specify the service throttling behaviour as an attribute as well rather than having to rely on config files?
I can see the reasoning behind putting these in the config file so you don’t have to recompile and redistribute whenever you wishes to change the throttling behaviour, but there are certain edge cases where it warrants the use of such attribute. For instance, my WCF services are dependency injected through a custom wrapper and I want different throttling behaviour for each service rather than a carpet throttling behaviour which applies to all my services.

ServiceThrottlingAttribute

Whilst there’s no built-in ServiceThrottlingAttribute in the framework, you can easily write one yourself. Here’s one I wrote the other day based on Ralph Squillace’s code sample here:
I made some modification so you don’t HAVE TO specify the value for each type of throttling and will use the default values where possible.
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
public sealed class ServiceThrottlingAttribute : Attribute, IServiceBehavior
{
    public int MaxConcurrentCalls { get; set; }
    public int MaxConcurrentInstances { get; set; }
    public int MaxConcurrentSessions { get; set; }
    public void Validate(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
    }
    public void AddBindingParameters(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase, Collection<ServiceEndpoint> endpoints, BindingParameterCollection bindingParameters)
    {
    }
    public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
        var currentThrottle = serviceDescription.Behaviors.Find<ServiceThrottlingBehavior>();
        if (currentThrottle == null)
        {
            serviceDescription.Behaviors.Add(GetConfiguredServiceThrottlingBehaviour());
        }
    }

    private ServiceThrottlingBehavior GetConfiguredServiceThrottlingBehaviour()
    {
        var behaviour = new ServiceThrottlingBehavior();
        if (MaxConcurrentCalls > 0)
        {
            behaviour.MaxConcurrentCalls = MaxConcurrentCalls;
        }
        if (MaxConcurrentInstances > 0)
        {
            behaviour.MaxConcurrentInstances = MaxConcurrentInstances;
        }
        if (MaxConcurrentSessions > 0)
        {
            behaviour.MaxConcurrentSessions = MaxConcurrentSessions;
        }

        return behaviour;
    }
}

Performance

Before you go ahead and create yourself an ultra-scalable service by setting the max concurrent calls/sessions/instances values to be very high you need to be aware what the performance implications are.

First, you need to know that WCF is configured to be safe from DOS attacks out of the box and the default configurations for service throttling behaviour is very conservative:

  1. MaxConcurrentCalls – default is 16, represents the max number of messages that can actively be processed.
  2. MaxConcurrentInstances – default is 26. For a “PerSession” service this represents the max number of session; for a “PerCall” service this represents the max number of concurrent calls; for a “Singleton” service this is meaningless.
  3. MaxConcurrentSessions – default is 10, represents the max number of sessions a service can accept at one time and only affects session-enabled channels. Increase this to allow more than 10 concurrent sessions.
Prevention of DOS attacks aside, each concurrent request your service is processing also requires at least a new thread to carry out the task. Whilst I don’t know whether WCF uses the framework ThreadPool (which also has further performance implications, see here), there is a balance between the number of concurrent threads and how much time the CPU spends on context switching between the different threads. Too many concurrent threads and the performance starts to suffer as well as the response times, too few concurrent threads and the CPU is under utilised, the response times suffer and more requests are timed out as they wait in the queue.
According to Dan Rigsby’s blog post on throttling WCF services (see references section below), the recommendations for each type of service is as follows:

 

If your InstanceContext is set to “PerCall” you should set maxConcurrentSessions and maxConcurrentCalls to the same value since each call is its own session.  You should target this value to be at least 25-30.  However you shouldn’t need to go higher than 64.

If your InstanceContext is set to “PerSession” you should set maxConcurrentCalls to be at least 25-30.  Your maxConcurrentSessions should be the number of users you want to have concurrently connected.

If your InstanceContext is set to “Single” you should use the same settings as “PerSession”, but the maxConcurrentSessions will only apply if session support is active (this is set with the SessionMode attribute on the ServiceContractAttribute).

References

Dan Rigsby’s blog post on thorttling WCF services

.Net Tips – using configSource or file attribute to externalize config sections

There’s a little known attribute which was introduced in the .Net framework 2.0 called configSource, which allows you to externalize sections of the configuration file.

It works just like the optional file attribute found on the appSettings element:

<appSettings file = “relative file name” />

but can be added to any configuration section to specify an external file for that section:

<CustomSection configSource=”relative file name” />

When it comes to WCF though, unfortunately the configSource attribute doesn’t work on the <system.serviceModel> section group, but luckily you can still apply it on the child config sections. See the reference link below for some examples.

Reference:

http://weblogs.asp.net/cibrax/archive/2007/07/24/configsource-attribute-on-system-servicemodel-section.aspx

DataContract Serialization by Reference using the IsReference Property

I came across this blog post the other day which introduced to me a cool addition to the DataContract serializer – the ability to generate XML by reference rather than by value:

http://www.zamd.net/2008/05/20/DataContractSerializerAndIsReferenceProperty.aspx

Not much for me to add to it really, just read the blog to see how it works.

Dealing with Circular References in WCF

Using entity classes in your application and WCF is complaining about the circular references between your classes? Well, I had the exact same problem not long ago, and I found this post on James Kovac’s blog about circular references and how to get around them:

http://www.jameskovacs.com/blog/GoingAroundInCirclesWithWCF.aspx

The key things to note from this post is that:

  1. WCF can handle circular references, but is not switched on by default
  2. There is a boolean flag in the constructor for the DataContractSerializer class which enables object references to be preserved
  3. We can tell WCF to switch this on by deriving from the DataContractSerializerOperationBehaviour class (as shown in the blog post above)

By this point you’re probably wondering why circular reference handling is not enabled by default, according to James Kovac, it’s because WCF by default tries to respect interoperability safety:

Now why can’t WCF handle circular references out-of-the-box. The reason is that there is no industry-accepted, interoperable way of expressing anything but parent-child relationships in XML. You can use the ID/IDREF feature of XML or the key/keyref feature of XML Schema, but a lot of serializers don’t respect these attributes or handle them properly. So if you want to serialize circular references, you need to stray out of the realm of safe interoperability.

So here are the classes you need to create to extend DataContractSerializerOperationBehaviour in order to preserve object reference:

public class PreserveReferencesOperationBehavior : DataContractSerializerOperationBehavior
{
     public PreserveReferencesOperationBehavior(OperationDescription operation) : base(operation)
     {
     }

     public PreserveReferencesOperationBehavior(
          OperationDescription operation, DataContractFormatAttribute dataContractFormatAttribute)
          : base(operation, dataContractFormatAttribute)
     {
     }

     public override XmlObjectSerializer CreateSerializer(
          Type type, XmlDictionaryString name, XmlDictionaryString ns, IList<Type> knownTypes)
     {
          return new DataContractSerializer(type, name, ns, knownTypes,
                                            0x7FFF /*maxItemsInObjectGraph*/,
                                            false/*ignoreExtensionDataObject*/,
                                            true/*preserveObjectReferences*/,
                                            null/*dataContractSurrogate*/);
     }
}

And the attribute to use on your operation contract:

public class PreserveReferencesAttribute : Attribute, IOperationBehavior
{
     public void AddBindingParameters(OperationDescription description,
                                      BindingParameterCollection parameters)
     {
     }

     public void ApplyClientBehavior(OperationDescription description, ClientOperation proxy)
     {
          IOperationBehavior innerBehavior = new PreserveReferencesOperationBehavior(description);
          innerBehavior.ApplyClientBehavior(description, proxy);
     }

     public void ApplyDispatchBehavior(OperationDescription description,
                                       DispatchOperation dispatch)
     {
          IOperationBehavior innerBehavior = new PreserveReferencesOperationBehavior(description);
          innerBehavior.ApplyDispatchBehavior(description, dispatch);
     }

     public void Validate(OperationDescription description)
     {
     }
}

which you apply like this:

[OperationContract]
[PreserveReferences]
MyClass RetrieveMyClass();