Threading – introducing SmartThreadPool

Yan Cui

I help clients go faster for less using serverless technologies.

This article is brought to you by

Is your CI build step taking too long? Try Depot for free today and experience up to 40x faster build speed!

Unlock faster CI for FREE

As I’ve mentioned in my previous post, the biggest problem with using the .Net ThreadPool is that there’s a limited number of threads in the pool and you’ll be sharing with other .Net framework classes so you need to be on your best behaviour and not hog the thread pool with long running or blocking jobs.

But what if you need to do lots of concurrent IO jobs which blocks and want your main thread to wait till all these threads are done? Normally in these cases you will need to create your own threads, start them and then join all the spawned threads with your main thread.

This approach would of course carry with it the overhead of creating and destroying threads, and if you’re doing the same thing in lots of different places simultaneously it can also push your CPU to 100% too. For example, you’ve got multiple threads running, and each spawns many more threads to do their concurrent IO jobs at the same time.

In situations like this, you almost want to have a thread pool for these jobs which is separate from the .Net ThreadPool, that way you avoid the overheads of using your own threads and can curb the CPU usage because you’re not creating new threads unnecessarily. But to create your own implementation that’s anywhere near as good as the .Net ThreadPool is no small undertaking, which is why I was so glad when I found out about SmartThreadPool.

Here are some of the features which I found really useful:

SmartThreadPool objects are instantiable

Which means you can create different thread pools for different type of jobs, each with an appropriate number of threads. This way each type of jobs have its own dedicated pool of threads and won’t eat into each other’s quota (and that of the .Net framework classes!).

Work items can have a return value, and exceptions are passed back to the caller

Getting return values from thread pool threads has always been a pain, as is catching any exceptions that are thrown on those threads, and with the SmartThreadPool you can now do both!

// create new SmartThreadPool
var threadPool = new SmartThreadPool();
// queue up work items
var result = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => true), new object());
var exception = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => { throw new Exception(); }), new object());

// wait till the items are done
if (SmartThreadPool.WaitAll(new[] { result, exception }))
    Console.WriteLine(result.Result); // prints true
        Console.WriteLine(exception.Result); // throws exception
    catch (Exception)

Work items can have priority

The SmartThreadPool allows you to specify the priority of the threads in the pool, so it’s possible to have a thread pool for critical jobs with higher priority and a separate thread pool for non-essential jobs which have a lower priority:

// create a STPStartInfo object and change the default values
var stpStartInfo = new STPStartInfo
        DisposeOfStateObjects = true,
        MinWorkerThreads = 0,
        MaxWorkerThreads = 10,
        ThreadPriority = ThreadPriority.Lowest
// create the SmartThreadPool instance
var threadPool = new SmartThreadPool(stpStartInfo);


SmartThreadPool’s CodePlex homepage

MSDN article on managing the thread pool

Whenever you’re ready, here are 4 ways I can help you:

  1. Production-Ready Serverless: Join 20+ AWS Heroes & Community Builders and 1000+ other students in levelling up your serverless game. This is your one-stop shop for quickly levelling up your serverless skills.
  2. Do you want to know how to test serverless architectures with a fast dev & test loop? Check out my latest course, Testing Serverless Architectures and learn the smart way to test serverless.
  3. I help clients launch product ideas, improve their development processes and upskill their teams. If you’d like to work together, then let’s get in touch.
  4. Join my community on Discord, ask questions, and join the discussion on all things AWS and Serverless.

Leave a Comment

Your email address will not be published. Required fields are marked *