Threading — thread-safe size capped queue

Here is a queue class based on the imple­men­ta­tion Marc Grav­ell pro­vided in this Stack­Over­flow question:

/// <summary>
/// A thread-safe fixed sized queue implementation
/// See: http://stackoverflow.com/questions/530211/creating-a-blocking-queuet-in-net/530228#530228
/// </summary>
public sealed class SizeQueue<T>
{
    private readonly Queue<T> _queue = new Queue<T>();
    private readonly int _maxSize;
    private readonly object _syncRoot = new object();
    public SizeQueue(int maxSize)
    {
        _maxSize = maxSize;
    }
    public int Count
    {
        get
        {
            lock (_syncRoot)
            {
                return _queue.Count;
            }
        }
    }
    public object SyncRoot
    {
        get
        {
            return _syncRoot;
        }
    }
    /// <summary>
    /// Puts an item onto the queue
    /// </summary>
    public void Enqueue(T item)
    {
        lock (_syncRoot)
        {
            // don't enqueue new item if the max size has been met
            while (_queue.Count >= _maxSize)
            {
                Monitor.Wait(_syncRoot);
            }
            _queue.Enqueue(item);
            // wake up any blocked dequeue
            Monitor.PulseAll(_syncRoot);
        }
    }
    /// <summary>
    /// Returns the first item from the queue
    /// </summary>
    public T Dequeue()
    {
        return Dequeue(1).FirstOrDefault();
    }
    /// <summary>
    /// Returns the requested number of items from the head of the queue
    /// </summary>
    public IEnumerable<T> Dequeue(int count)
    {
        lock (_syncRoot)
        {
            // wait until there're items on the queue
            while (_queue.Count == 0)
            {
                Monitor.Wait(_syncRoot);
            }
            
            // read as many items off the queue as required (and possible)
            var items = new List<T>();
            while (count > 0 && _queue.Count > 0)
            {
                items.Add(_queue.Dequeue());
                count--;
            }
           return items;
        }
    }
}
Share
May 18, 2010 | | Tags: , , , | 1 Comment »

HornGet — a place to get your OpenSource fix

Whilst lis­ten­ing to the lat­est .Net Rocks! pod­cast, I stum­bled across HornGet, which is a site that pro­vides pre-compiled bina­ries for well known open source projects such as NHiber­nate, Rhi­noMocks, etc.

I posted a while back about the prob­lems I had find­ing a build of NHiber­nate which works with both Flu­entNHiber­nate and Linq2NHibernate, only if I knew about HornGet back then!

There are about 30 odd projects hosted on there for now, sep­a­rated into a num­ber of cat­e­gories such as ORM, Log­ger, etc.

Share
| | Tags: , , | No Comments »

Anders Hejlsberg’s podcast session with .Net Rocks!

Hav­ing just lis­tened to a recent .Net Rocks! pod­cast with Anders Hejls­berg (chief archi­tect of the C# lan­guage) in a fit­tingly named show, “Anders Hejls­berg blows our mind!”, I felt it worth­while to note down some of the views Anders shared with us, and some nice quotes for you to use in your next geek con­ver­sa­tion :-P

On LINQ

Recent changes to C# lan­guage have intro­duced more func­tional and dynamic fea­tures to a tra­di­tion­ally imper­a­tive and sta­t­i­cally typed language.

Lan­guage exten­sions such as LINQ intro­duced an iso­lated set of func­tional fea­tures (the intro­duc­tion of the LINQ query lan­guage for exam­ple) and through its dif­fer­ent flavours (Linq2SQL, Linq2XML and Linq2Objects) allows for a uni­fied way of work­ing with data – some­thing that has been lost since the days of old when con­cerns such as scal­a­bil­ity, trans­ac­tions, etc. drove a break­away of data man­age­ment capa­bil­i­ties into the Rela­tional Data­base realm, away from the land of gen­eral pur­pose pro­gram­ming languages.

Besides being a data query lan­guage, LINQ is also a data trans­for­ma­tion lan­guage allow­ing for the trans­for­ma­tion of data between objects and XML, or SQL. It also moves the level of abstrac­tion upwards, allow­ing devel­op­ers to work on a higher level and say more about the ‘what’ rather than the ‘how’. This ele­va­tion of the abstrac­tion level is impor­tant because it lends a greater sup­port for a con­cur­rent exe­cu­tion envi­ron­ment – PLINQ for instance allows for par­al­lel exe­cu­tion of LINQ queries on mul­ti­ple cores.

On Par­al­lelism

In the years gone by, the hard­ware evo­lu­tion has pro­vided for a quick and easy way to solve our soft­ware per­for­mance prob­lems, as CPUs’ clock speeds go up they pro­vide an all-around increase in per­for­mance to all our soft­ware. How­ever, the increase in clock speed has dried up because we have hit the phys­i­cal limit as CPUs get too hot, and so began the cur­rent cycle of multi-core CPUs.

This presents a dif­fer­ent chal­lenge for soft­ware devel­op­ers as they can no longer just rely on the hard­ware to solve per­for­mance issues and have to be smart to lever­age off the poten­tials of the new gen­er­a­tion of CPUs:

“The free lunch is over, now you really have to learn to cook!”

Whilst con­cur­rent pro­gram­ming is hard, the lan­guage is try­ing to make it eas­ier by intro­duc­ing PLINQ and the Task Par­al­lel Library (TPL), and Anders fore­sees future changes which will make it eas­ier to cre­ate immutable types which work bet­ter in a con­cur­rent envi­ron­ment (see my post on immutable structs here). But bear in mind that not all things can be par­al­lelised and even those that can be par­al­lelise they don’t always guar­an­tee a lin­ear gain in performance.

The prob­lem with the exist­ing thread cre­ation func­tions in C# is that we have a fairly shal­low rep­re­sen­ta­tion of the OS resources, which are rel­a­tively expen­sive – they don’t con­sume a lot of phys­i­cal mem­ory but they con­sume lots of log­i­cal address space. By default, every time you cre­ate a thread a MB of log­i­cal address space is allo­cated to the thread, and with only 2–3 GB of avail­able log­i­cal space in a 32bit machine you’re only allowed a few thou­sand threads concurrently.

The TPL pro­vides a way for us to decou­ple our work from the threads, and enables us to rea­son about gen­er­at­ing log­i­cal units of work with­out hav­ing to worry about thread affinity.

On Func­tional Programming

“Func­tional pro­gram­ming is won­der­ful, except when you have to write real apps.”

In a pure func­tional pro­gram­ming lan­guage, there can be no side effects, but in any real-world appli­ca­tions there has to be side-effects some­where, writ­ing to a log file is a side effect and so is updat­ing a row in the Database.

So the trick is to iden­tify areas in your appli­ca­tion which doesn’t require side effects and write islands of func­tional code, and find ways to marry them to imper­a­tive code for their side effects. The great thing about no hav­ing any side effects is that you can rule out any con­sid­er­a­tion about state changes when you call a method, e.g. when you call Math.Round() on a num­ber you know what you’re gonna get and you never stop to think about the con­se­quences of call­ing it twice because there’re no side effects when you call the method and for the same argu­ment it will always return the same value.

On Dynamic Typ­ing and C# 5

In C# 4, the sup­port for dynamic typ­ing has been added to the lan­guage. This has been intro­duced very much with the chal­lenges devel­op­ers face from day to day in mind, because we work with non-schematised data struc­tures all the time, e.g. every time you work against a cloud data service.

And finally, briefly touch­ing on what’s to come in C# 5, Anders talked about the theme of C# 5 – meta pro­gram­ming. Some of the work he’s doing now is about open­ing up the com­piler, and pro­vid­ing the com­piler as a ser­vice. This will make it pos­si­ble for us to break away from the tra­di­tional com­piler model of work­ing with source files, and enable us to have inter­ac­tive prompts and REPL like the F# Inter­ac­tive win­dow that is avail­able for F#. This also opens the door for meta pro­gram­ming capa­bil­ity, pro­grams that writes itself, a fea­ture that has dri­ven the suc­cess of Ruby and Ruby on Rails.

Share
| | Tags: , , | 1 Comment »

.Net Tips — Using custom ServiceThrottlingAttribute to specify WCF service throttling behaviour

If you have cre­ated a WCF ser­vice in the past then I assume you’re aware that WCF is very heav­ily configuration-driven and that you can spec­ify the ser­vice behav­iour includ­ing the throt­tling para­me­ters (Max­Con­cur­rent­Calls, Max­Con­cur­rentIn­stances, Max­Con­cur­rentSes­sions) in the con­fig file.
But to spec­ify the type of ser­vice (Per­Call, PerS­es­sion or Sin­gle­ton) you need to apply the Ser­vice­Be­hav­iour attribute to your type.
Now, wouldn’t it be nice if you can spec­ify the ser­vice throt­tling behav­iour as an attribute as well rather than hav­ing to rely on con­fig files?
I can see the rea­son­ing behind putting these in the con­fig file so you don’t have to recom­pile and redis­trib­ute when­ever you wishes to change the throt­tling behav­iour, but there are cer­tain edge cases where it war­rants the use of such attribute. For instance, my WCF ser­vices are depen­dency injected through a cus­tom wrap­per and I want dif­fer­ent throt­tling behav­iour for each ser­vice rather than a car­pet throt­tling behav­iour which applies to all my services.

Ser­viceThrot­tlin­gAt­tribute

Whilst there’s no built-in Ser­viceThrot­tlin­gAt­tribute in the frame­work, you can eas­ily write one your­self. Here’s one I wrote the other day based on Ralph Squillace’s code sam­ple here:
I made some mod­i­fi­ca­tion so you don’t HAVE TO spec­ify the value for each type of throt­tling and will use the default val­ues where possible.
[AttributeUsage(AttributeTargets.Class, AllowMultiple = false)]
public sealed class ServiceThrottlingAttribute : Attribute, IServiceBehavior
{
    public int MaxConcurrentCalls { get; set; }
    public int MaxConcurrentInstances { get; set; }
    public int MaxConcurrentSessions { get; set; }
    public void Validate(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
    }
    public void AddBindingParameters(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase, Collection<ServiceEndpoint> endpoints, BindingParameterCollection bindingParameters)
    {
    }
    public void ApplyDispatchBehavior(ServiceDescription serviceDescription, ServiceHostBase serviceHostBase)
    {
        var currentThrottle = serviceDescription.Behaviors.Find<ServiceThrottlingBehavior>();
        if (currentThrottle == null)
        {
            serviceDescription.Behaviors.Add(GetConfiguredServiceThrottlingBehaviour());
        }
    }

    private ServiceThrottlingBehavior GetConfiguredServiceThrottlingBehaviour()
    {
        var behaviour = new ServiceThrottlingBehavior();
        if (MaxConcurrentCalls > 0)
        {
            behaviour.MaxConcurrentCalls = MaxConcurrentCalls;
        }
        if (MaxConcurrentInstances > 0)
        {
            behaviour.MaxConcurrentInstances = MaxConcurrentInstances;
        }
        if (MaxConcurrentSessions > 0)
        {
            behaviour.MaxConcurrentSessions = MaxConcurrentSessions;
        }

        return behaviour;
    }
}

Per­for­mance

Before you go ahead and cre­ate your­self an ultra-scalable ser­vice by set­ting the max con­cur­rent calls/sessions/instances val­ues to be very high you need to be aware what the per­for­mance impli­ca­tions are.

First, you need to know that WCF is con­fig­ured to be safe from DOS attacks out of the box and the default con­fig­u­ra­tions for ser­vice throt­tling behav­iour is very conservative:

  1. Max­Con­cur­rent­Calls - default is 16, rep­re­sents the max num­ber of mes­sages that can actively be processed.
  2. Max­Con­cur­rentIn­stances - default is 26. For a “PerS­es­sion” ser­vice this rep­re­sents the max num­ber of ses­sion; for a “Per­Call” ser­vice this rep­re­sents the max num­ber of con­cur­rent calls; for a “Sin­gle­ton” ser­vice this is meaningless.
  3. Max­Con­cur­rentSes­sions - default is 10, rep­re­sents the max num­ber of ses­sions a ser­vice can accept at one time and only affects session-enabled chan­nels. Increase this to allow more than 10 con­cur­rent sessions.
Pre­ven­tion of DOS attacks aside, each con­cur­rent request your ser­vice is pro­cess­ing also requires at least a new thread to carry out the task. Whilst I don’t know whether WCF uses the frame­work Thread­Pool (which also has fur­ther per­for­mance impli­ca­tions, see here), there is a bal­ance between the num­ber of con­cur­rent threads and how much time the CPU spends on con­text switch­ing between the dif­fer­ent threads. Too many con­cur­rent threads and the per­for­mance starts to suf­fer as well as the response times, too few con­cur­rent threads and the CPU is under utilised, the response times suf­fer and more requests are timed out as they wait in the queue.
Accord­ing to Dan Rigsby’s blog post on throt­tling WCF ser­vices (see ref­er­ences sec­tion below), the rec­om­men­da­tions for each type of ser­vice is as follows:

If your Instance­Con­text is set to “Per­Call” you should set max­Con­cur­rentSes­sions and max­Con­cur­rent­Calls to the same value since each call is its own ses­sion.  You should tar­get this value to be at least 25–30.  How­ever you shouldn’t need to go higher than 64.

If your Instance­Con­text is set to “PerS­es­sion” you should set max­Con­cur­rent­Calls to be at least 25–30.  Your max­Con­cur­rentSes­sions should be the num­ber of users you want to have con­cur­rently connected.

If your Instance­Con­text is set to “Sin­gle” you should use the same set­tings as “PerS­es­sion”, but the max­Con­cur­rentSes­sions will only apply if ses­sion sup­port is active (this is set with the Ses­sion­Mode attribute on the Ser­vice­Con­trac­tAt­tribute).

Ref­er­ences

Dan Rigsby’s blog post on thort­tling WCF services

Share
May 17, 2010 | | Tags: , , , , | No Comments »

Buzzword Buster – Technical Debt

Yet another recur­ring phrase on DotNetRocks’s pod­casts, and a catchy one at that! The term ‘tech­ni­cal debt’ was coined by Ward Cun­ning­ham to describe the even­tual con­se­quences a soft­ware devel­op­ment orga­ni­za­tion incurs when it chooses to do things the quick and dirty way, which ben­e­fits the short-term but increases com­plex­ity and ulti­mately is more costly in the long term.

Tech­ni­cal debt incurs inter­est pay­ments in the shape of extra effort required for future devel­op­ment, but can be paid down by refac­tor­ing the quick and dirty design into bet­ter designs. It costs to pay down tech­ni­cal debt, but you gain by reduc­ing future inter­est payments.

Tech­ni­cal debt is almost unavoid­able in any real-world devel­op­ment project, and some­times it may be sen­si­ble to incur some tech­ni­cal debt in order to meet an impor­tant dead­line just as busi­nesses incur some debt to take advan­tage of a mar­ket opportunity.

Ref­er­ences:

Steve McConnell’s in-depth look on tech­ni­cal debt

Cod­ing Horror’s arti­cle on pay­ing down your tech­ni­cal debt

Share
May 16, 2010 | | Tags: , | No Comments »
« Older Entries
Newer Entries »