Threading – introducing SmartThreadPool

Check out my new course Learn you some Lambda best practice for great good! and learn the best practices for performance, cost, security, resilience, observability and scalability.

As I’ve mentioned in my previous post, the biggest problem with using the .Net ThreadPool is that there’s a limited number of threads in the pool and you’ll be sharing with other .Net framework classes so you need to be on your best behaviour and not hog the thread pool with long running or blocking jobs.

But what if you need to do lots of concurrent IO jobs which blocks and want your main thread to wait till all these threads are done? Normally in these cases you will need to create your own threads, start them and then join all the spawned threads with your main thread.

This approach would of course carry with it the overhead of creating and destroying threads, and if you’re doing the same thing in lots of different places simultaneously it can also push your CPU to 100% too. For example, you’ve got multiple threads running, and each spawns many more threads to do their concurrent IO jobs at the same time.

In situations like this, you almost want to have a thread pool for these jobs which is separate from the .Net ThreadPool, that way you avoid the overheads of using your own threads and can curb the CPU usage because you’re not creating new threads unnecessarily. But to create your own implementation that’s anywhere near as good as the .Net ThreadPool is no small undertaking, which is why I was so glad when I found out about SmartThreadPool.

Here are some of the features which I found really useful:

SmartThreadPool objects are instantiable

Which means you can create different thread pools for different type of jobs, each with an appropriate number of threads. This way each type of jobs have its own dedicated pool of threads and won’t eat into each other’s quota (and that of the .Net framework classes!).

Work items can have a return value, and exceptions are passed back to the caller

Getting return values from thread pool threads has always been a pain, as is catching any exceptions that are thrown on those threads, and with the SmartThreadPool you can now do both!

// create new SmartThreadPool
var threadPool = new SmartThreadPool();
// queue up work items
var result = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => true), new object());
var exception = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => { throw new Exception(); }), new object());

// wait till the items are done
if (SmartThreadPool.WaitAll(new[] { result, exception }))
    Console.WriteLine(result.Result); // prints true
        Console.WriteLine(exception.Result); // throws exception
    catch (Exception)

Work items can have priority

The SmartThreadPool allows you to specify the priority of the threads in the pool, so it’s possible to have a thread pool for critical jobs with higher priority and a separate thread pool for non-essential jobs which have a lower priority:

// create a STPStartInfo object and change the default values
var stpStartInfo = new STPStartInfo
        DisposeOfStateObjects = true,
        MinWorkerThreads = 0,
        MaxWorkerThreads = 10,
        ThreadPriority = ThreadPriority.Lowest
// create the SmartThreadPool instance
var threadPool = new SmartThreadPool(stpStartInfo);


SmartThreadPool’s CodePlex homepage

MSDN article on managing the thread pool

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my newsletter

Hi, I’m Yan. I’m an AWS Serverless Hero and the author of Production-Ready Serverless.

I specialise in rapidly transitioning teams to serverless and building production-ready services on AWS.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.

Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.

Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!

Are you working with Serverless and looking for expert training to level-up your skills? Or are you looking for a solid foundation to start from? Look no further, register for my Production-Ready Serverless workshop to learn how to build production-grade Serverless applications!

Find a workshop near you