Threading – introducing SmartThreadPool

As I’ve mentioned in my previous post, the biggest problem with using the .Net ThreadPool is that there’s a limited number of threads in the pool and you’ll be sharing with other .Net framework classes so you need to be on your best behaviour and not hog the thread pool with long running or blocking jobs.

But what if you need to do lots of concurrent IO jobs which blocks and want your main thread to wait till all these threads are done? Normally in these cases you will need to create your own threads, start them and then join all the spawned threads with your main thread.

This approach would of course carry with it the overhead of creating and destroying threads, and if you’re doing the same thing in lots of different places simultaneously it can also push your CPU to 100% too. For example, you’ve got multiple threads running, and each spawns many more threads to do their concurrent IO jobs at the same time.

In situations like this, you almost want to have a thread pool for these jobs which is separate from the .Net ThreadPool, that way you avoid the overheads of using your own threads and can curb the CPU usage because you’re not creating new threads unnecessarily. But to create your own implementation that’s anywhere near as good as the .Net ThreadPool is no small undertaking, which is why I was so glad when I found out about SmartThreadPool.

Here are some of the features which I found really useful:

SmartThreadPool objects are instantiable

Which means you can create different thread pools for different type of jobs, each with an appropriate number of threads. This way each type of jobs have its own dedicated pool of threads and won’t eat into each other’s quota (and that of the .Net framework classes!).

Work items can have a return value, and exceptions are passed back to the caller

Getting return values from thread pool threads has always been a pain, as is catching any exceptions that are thrown on those threads, and with the SmartThreadPool you can now do both!

// create new SmartThreadPool
var threadPool = new SmartThreadPool();
// queue up work items
var result = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => true), new object());
var exception = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => { throw new Exception(); }), new object());

// wait till the items are done
if (SmartThreadPool.WaitAll(new[] { result, exception }))
{
    Console.WriteLine(result.Result); // prints true
    try
    {
        Console.WriteLine(exception.Result); // throws exception
    }
    catch (Exception)
    {
        Console.WriteLine("Exception");
    }
}

Work items can have priority

The SmartThreadPool allows you to specify the priority of the threads in the pool, so it’s possible to have a thread pool for critical jobs with higher priority and a separate thread pool for non-essential jobs which have a lower priority:

// create a STPStartInfo object and change the default values
var stpStartInfo = new STPStartInfo
    {
        DisposeOfStateObjects = true,
        MinWorkerThreads = 0,
        MaxWorkerThreads = 10,
        ThreadPriority = ThreadPriority.Lowest
    };
// create the SmartThreadPool instance
var threadPool = new SmartThreadPool(stpStartInfo);

References:

SmartThreadPool’s CodePlex homepage

MSDN article on managing the thread pool