Threading — introducing SmartThreadPool

As I’ve men­tioned in my pre­vi­ous post, the biggest prob­lem with using the .Net Thread­Pool is that there’s a lim­it­ed num­ber of threads in the pool and you’ll be shar­ing with oth­er .Net frame­work class­es so you need to be on your best behav­iour and not hog the thread pool with long run­ning or block­ing jobs.

But what if you need to do lots of con­cur­rent IO jobs which blocks and want your main thread to wait till all these threads are done? Nor­mal­ly in these cas­es you will need to cre­ate your own threads, start them and then join all the spawned threads with your main thread.

This approach would of course car­ry with it the over­head of cre­at­ing and destroy­ing threads, and if you’re doing the same thing in lots of dif­fer­ent places simul­ta­ne­ous­ly it can also push your CPU to 100% too. For exam­ple, you’ve got mul­ti­ple threads run­ning, and each spawns many more threads to do their con­cur­rent IO jobs at the same time.

In sit­u­a­tions like this, you almost want to have a thread pool for these jobs which is sep­a­rate from the .Net Thread­Pool, that way you avoid the over­heads of using your own threads and can curb the CPU usage because you’re not cre­at­ing new threads unnec­es­sar­i­ly. But to cre­ate your own imple­men­ta­tion that’s any­where near as good as the .Net Thread­Pool is no small under­tak­ing, which is why I was so glad when I found out about Smart­Thread­Pool.

Here are some of the fea­tures which I found real­ly use­ful:

Smart­Thread­Pool objects are instan­tiable

Which means you can cre­ate dif­fer­ent thread pools for dif­fer­ent type of jobs, each with an appro­pri­ate num­ber of threads. This way each type of jobs have its own ded­i­cat­ed pool of threads and won’t eat into each other’s quo­ta (and that of the .Net frame­work class­es!).

Work items can have a return val­ue, and excep­tions are passed back to the caller

Get­ting return val­ues from thread pool threads has always been a pain, as is catch­ing any excep­tions that are thrown on those threads, and with the Smart­Thread­Pool you can now do both!

// create new SmartThreadPool
var threadPool = new SmartThreadPool();
// queue up work items
var result = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => true), new object());
var exception = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => { throw new Exception(); }), new object());

// wait till the items are done
if (SmartThreadPool.WaitAll(new[] { result, exception }))
{
    Console.WriteLine(result.Result); // prints true
    try
    {
        Console.WriteLine(exception.Result); // throws exception
    }
    catch (Exception)
    {
        Console.WriteLine("Exception");
    }
}

Work items can have pri­or­i­ty

The Smart­Thread­Pool allows you to spec­i­fy the pri­or­i­ty of the threads in the pool, so it’s pos­si­ble to have a thread pool for crit­i­cal jobs with high­er pri­or­i­ty and a sep­a­rate thread pool for non-essen­tial jobs which have a low­er pri­or­i­ty:

// create a STPStartInfo object and change the default values
var stpStartInfo = new STPStartInfo
    {
        DisposeOfStateObjects = true,
        MinWorkerThreads = 0,
        MaxWorkerThreads = 10,
        ThreadPriority = ThreadPriority.Lowest
    };
// create the SmartThreadPool instance
var threadPool = new SmartThreadPool(stpStartInfo);

References:

SmartThreadPool’s Code­Plex home­page

MSDN arti­cle on man­ag­ing the thread pool