You can become a serverless blackbelt. Enrol in my course Learn you some Lambda best practice for great good! and learn best practices for performance, cost, security, resilience, observability and scalability. By the end of this course, you should be able to make informed decisions on which AWS service to use with Lambda and how to build highly scalable, resilient and cost efficient serverless applications.
There are a lot of discussions on the pros and cons of using the ThreadPool and creating your own threads. Having spent a bit of time reading what others have to say, here’s a summary of the things I’ve picked up on.
The problem with creating your own threads
Creating and destroying threads has a high CPU usage, so when you need to perform lots of small, simple tasks concurrently the overhead of creating your own threads can take up a significant portion of the CPU cycles and severely affect the final response time. This is especially true in stress conditions where executing multiple threads can push CPU to 100% and most of the time would be wasted in context switching (swapping threads in and out of the processor along with their memory).
Using the Thread Pool
This is where the .Net Thread Pool comes in, where a number of threads are created ahead of time and kept around to pick up any work items you give them to do, without the overhead associated with creating your own threads.
When not to use the Thread Pool
In an ideal world you would always want to use the Thread Pool, but there are some real-world limitations. Most importantly, and the reason why most experts would tell you not to use the Thread Pool except for brief jobs is that: there is a limited number of threads in the .Net Thread Pool (250 per CPU by default), and they are being used by many of the .Net framework classes (e.g. timer events are fired on thread pool threads) so you wouldn’t want your application to hog the thread pool.
There are also a number of situations where you shouldn’t use the thread pool:
- You require a foreground thread, all the thread pool threads are background threads
- You require a thread to have a particular priority.
- You have tasks that cause the thread to block for long periods of time. The thread pool has a maximum number of threads, so a large number of blocked thread pool threads might prevent tasks from starting.
- You need to place threads into a single-threaded apartment. All ThreadPool threads are in the multithreaded apartment.
- You need to have a stable identity associated with the thread, or to dedicate a thread to a task.
Exceptions in Thread Pool threads
Unhandled exceptions on thread pool threads terminate the process with 3 exceptions:
- A ThreadAbortException is thrown in a thread pool thread, because Abort was called.
- An AppDomainUnloadedException is thrown in a thread pool thread, because the application domain is being unloaded.
- The common language runtime (CLR) or a host process terminates the thread.
When to create your own threads
As I’ve mentioned already, creating your own threads is bad when lots of simple tasks require a relative large overhead in context switching, and the Thread Pool is bad for long running, or blocking tasks. Which leads to the natural conclusion :-P – create your own threads for long running, or blocking tasks!
When working with the Thread Pool there are some useful methods at your disposable, including:
- GetAvailableThreads method which returns the number of threads available to you
- GetMinThreads method returns the number of idle threads the thread pool maintains in anticipation of new requests
- GetMaxThreads method returns the max number of thread pool threads that can be active concurrently
- SetMinThreads method sets the number of idle threads the thread pool maintains in anticipation of new requests
- SetMaxThreads method sets the number of thread pool threads that can be active concurrently
If you’re interested in how the ThreadPool class dynamically manages the size of the thread pool under the hood (despite giving you the option to set min and max threads) you should have a read of Pedram Razai’s blog post in the reference section.
And before you go, I mentioned earlier that all Thread Pool threads are background threads, so how do they differ from foreground threads? Well, foreground and Background threads are identical with one exception: a background thread does not keep the managed execution environment running. Once all foreground threads have been stopped in a managed process (where the .exe file is a managed assembly), the system stops all background threads and shuts down.
I specialise in rapidly transitioning teams to serverless and building production-ready services on AWS.
Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.
Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.
Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!
Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!
Are you working with Serverless and looking for expert training to level-up your skills? Or are you looking for a solid foundation to start from? Look no further, register for my Production-Ready Serverless workshop to learn how to build production-grade Serverless applications!
Here is a complete list of all my posts on serverless and AWS Lambda. In the meantime, here are a few of my most popular blog posts.
- Lambda optimization tip – enable HTTP keep-alive
- You are wrong about serverless and vendor lock-in
- You are thinking about serverless costs all wrong
- Just how expensive is the full AWS SDK?
- Many faced threats to Serverless security
- We can do better than percentile latencies
- Yubl’s road to Serverless
- AWS Lambda – should you have few monolithic functions or many single-purposed functions?
- AWS Lambda – compare coldstart time with different languages, memory and code sizes
- Guys, we’re doing pagination wrong
- Top 10 Serverless framework best practices