Threading – introducing SmartThreadPool

You can become a serverless blackbelt. Enrol to my 4-week online workshop Production-Ready Serverless and gain hands-on experience building something from scratch using serverless technologies. At the end of the workshop, you should have a broader view of the challenges you will face as your serverless architecture matures and expands. You should also have a firm grasp on when serverless is a good fit for your system as well as common pitfalls you need to avoid. Sign up now and get 15% discount with the code yanprs15!

As I’ve mentioned in my previous post, the biggest problem with using the .Net ThreadPool is that there’s a limited number of threads in the pool and you’ll be sharing with other .Net framework classes so you need to be on your best behaviour and not hog the thread pool with long running or blocking jobs.

But what if you need to do lots of concurrent IO jobs which blocks and want your main thread to wait till all these threads are done? Normally in these cases you will need to create your own threads, start them and then join all the spawned threads with your main thread.

This approach would of course carry with it the overhead of creating and destroying threads, and if you’re doing the same thing in lots of different places simultaneously it can also push your CPU to 100% too. For example, you’ve got multiple threads running, and each spawns many more threads to do their concurrent IO jobs at the same time.

In situations like this, you almost want to have a thread pool for these jobs which is separate from the .Net ThreadPool, that way you avoid the overheads of using your own threads and can curb the CPU usage because you’re not creating new threads unnecessarily. But to create your own implementation that’s anywhere near as good as the .Net ThreadPool is no small undertaking, which is why I was so glad when I found out about SmartThreadPool.

Here are some of the features which I found really useful:

SmartThreadPool objects are instantiable

Which means you can create different thread pools for different type of jobs, each with an appropriate number of threads. This way each type of jobs have its own dedicated pool of threads and won’t eat into each other’s quota (and that of the .Net framework classes!).

Work items can have a return value, and exceptions are passed back to the caller

Getting return values from thread pool threads has always been a pain, as is catching any exceptions that are thrown on those threads, and with the SmartThreadPool you can now do both!

// create new SmartThreadPool
var threadPool = new SmartThreadPool();
// queue up work items
var result = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => true), new object());
var exception = threadPool.QueueWorkItem(
    new Amib.Threading.Func<object, bool>(o => { throw new Exception(); }), new object());

// wait till the items are done
if (SmartThreadPool.WaitAll(new[] { result, exception }))
{
    Console.WriteLine(result.Result); // prints true
    try
    {
        Console.WriteLine(exception.Result); // throws exception
    }
    catch (Exception)
    {
        Console.WriteLine("Exception");
    }
}

Work items can have priority

The SmartThreadPool allows you to specify the priority of the threads in the pool, so it’s possible to have a thread pool for critical jobs with higher priority and a separate thread pool for non-essential jobs which have a lower priority:

// create a STPStartInfo object and change the default values
var stpStartInfo = new STPStartInfo
    {
        DisposeOfStateObjects = true,
        MinWorkerThreads = 0,
        MaxWorkerThreads = 10,
        ThreadPriority = ThreadPriority.Lowest
    };
// create the SmartThreadPool instance
var threadPool = new SmartThreadPool(stpStartInfo);

References:

SmartThreadPool’s CodePlex homepage

MSDN article on managing the thread pool

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my newsletter


Hi, I’m Yan. I’m an AWS Serverless Hero and I help companies go faster for less by adopting serverless technologies successfully.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.


Skill up your serverless game with this hands-on workshop.

My 4-week Production-Ready Serverless online workshop is back!

This course takes you through building a production-ready serverless web application from testing, deployment, security, all the way through to observability. The motivation for this course is to give you hands-on experience building something with serverless technologies while giving you a broader view of the challenges you will face as the architecture matures and expands.

We will start at the basics and give you a firm introduction to Lambda and all the relevant concepts and service features (including the latest announcements in 2020). And then gradually ramping up and cover a wide array of topics such as API security, testing strategies, CI/CD, secret management, and operational best practices for monitoring and troubleshooting.

If you enrol now you can also get 15% OFF with the promo code “yanprs15”.

Enrol now and SAVE 15%.


Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.


Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!


Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!