Check out my new course Learn you some Lambda best practice for great good! and learn the best practices for performance, cost, security, resilience, observability and scalability.
Hi, welcome to another weekly update.
You are wrong about serverless vendor lock-in. In this guest post for Lumigo, I challenge the debate on serverless and vendor lock-in. First of all, technology choices create coupling and moving cost but you’re never truly locked in as the option is always there to rewrite the application. Technology choices, including the adoption of serverless technologies, carry both risks and rewards. Serverless technologies allow your organization to remain focused on creating values for your business and your customers. In the increasingly competitive world we live in, lack of focus is a far bigger risk to any organization compared to being tightly coupled to one cloud provider. And lastly, the whole vendor lock-in FUD seems to be driven by vested interests, who are not looking out for what is best for us as customers!
Why your business needs serverless. In this guest post for the Jefferson Frank, I break down how developers, managers as well as business stakeholders can all benefit from serverless. For developers, serverless technologies help us build more scalable and resilient systems and let us focus on customer needs. For managers, they help my team respond to business needs faster and more cost efficiently. They allow my team to have more ownership and autonomy and reduce the amount of inter-team dependencies. As a business stakeholder, they make it possible for me to understand the operational cost of each transaction. Which allows me to make informed decisions on pricing as well as where (and when) we should optimize the system to maximise return-on-investment.
CloudFormation protip: use !Sub instead of !Join. If you’re working with CloudFormation, then you should use !Join only when you need to perform string concatenation. In every other case, !Sub is a better choice.
How to include Serverless Repository apps in serverless.yml. Here’s a simple workaround to let you include Serverless Repository apps when you’re using the Serverless framework.
AWS SAM + CloudFormation macros, a patch made in heaven. If you are using AWS SAM and find yourself constrained by the lack of a plugin system (like the Serverless framework), then you can customize/extend SAM using CloudFormation macros.
Lambda and Kinesis, beware of hot streams. In this guest post for Lumigo, I examined whether or not the scalability issues with having too many Lambda subscribers to a single Kinesis stream has been resolved with enhanced fan-out. Compared to 2017, things have improved a lot but a series issue still remains. After about 15 subscriber functions, it becomes almost impossible to add new subscribers because the deployment itself fails consistently. To wrap up the post, we also considered some workarounds for this issue.
There were some big announcements from AWS regarding Lambda.
Node 10 is now supported on Lambda. But there are a couple of things to note:
- It’s 10.x, so it’s pinned to the major version only. Which means AWS can update the minor versions at a later date without you having to make code changes. This should be a good thing, as we don’t miss out on new features from minor updates. But potentially it can also introduce bugs/subtle behaviour changes when the platform updates the minor version.
- The 10.x runtime runs on a brand new OS – Amazon Linux 2 (which was announced at re:invent 2018). The new base OS image is VERY LEAN – 800MB smaller than the existing base image (Amazon Linux 1). So a lot of the libs and binaries are not there anymore. If you call out to ps, tar, etc. or depend on libraries like bzip2, libcurl, etc. then you’ll need to pack those yourself going forward. If your function is pure JS then you should be fine. You can use tools like docker-lambda or SAM or Serverless framework (which supports invoke local with docker since v1.41.0) to check that your function still runs on the new OS image.
- If you are using extractedFields from CloudWatch Logs events for log parsing then Node 10.x prepends “INFO” or “ERROR” to the extracted event. I’d have liked it if it was introduced as a separate field, but it’s helpful to distinguish between stdout and stderr outputs. Specifically, when your function errors with an unhandled exception. It’s useful to get the extra context (that the message is for an error) when you’re parsing these log events.
- There is an open bug that function doesn’t start when the event loop is not empty. More details here.
For more details about Node 10.x, read this good thread by Michael Hart from Bustle. Despite all the excitement, I think you should hold off upgrading to the Node 10.x runtime just yet. At least until some of these early bugs/issues are ironed out.
Updates to all existing functions
AWS is rolling out an update to the underlying OS (Amazon Linux 1) to all non-Node 10.x Lambda functions over the next month or so. This shouldn’t affect most functions, unless you depend on native bins that are compiled against the specific OS version. More details, including a recommended migration plan, is available here.
AppSync supports multiple auth methods
You can now specify additional auth modes for different GraphQL actions. So now you can use a combination of:
- IAM auth for mutation actions, so they can be only triggered by your applications.
- API keys for queries so that client apps can access the GraphQL endpoint without having to go through Cognito/IAM.
I specialise in rapidly transitioning teams to serverless and building production-ready services on AWS.
Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.
Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. Enrol now and enjoy a special preorder price of £9.99 (~$13).
Are you working with Serverless and looking for expert training to level-up your skills? Or are you looking for a solid foundation to start from? Look no further, register for my Production-Ready Serverless workshop to learn how to build production-grade Serverless applications!
Here is a complete list of all my posts on serverless and AWS Lambda. In the meantime, here are a few of my most popular blog posts.
- Lambda optimization tip – enable HTTP keep-alive
- You are thinking about serverless costs all wrong
- Many faced threats to Serverless security
- We can do better than percentile latencies
- I’m afraid you’re thinking about AWS Lambda cold starts all wrong
- Yubl’s road to Serverless
- AWS Lambda – should you have few monolithic functions or many single-purposed functions?
- AWS Lambda – compare coldstart time with different languages, memory and code sizes
- Guys, we’re doing pagination wrong