Weekly update 26

Hi, welcome to another weekly update.

New posts

You are wrong about serverless vendor lock-in. In this guest post for Lumigo, I challenge the debate on serverless and vendor lock-in. First of all, technology choices create coupling and moving cost but you’re never truly locked in as the option is always there to rewrite the application. Technology choices, including the adoption of serverless technologies, carry both risks and rewards. Serverless technologies allow your organization to remain focused on creating values for your business and your customers. In the increasingly competitive world we live in, lack of focus is a far bigger risk to any organization compared to being tightly coupled to one cloud provider. And lastly, the whole vendor lock-in FUD seems to be driven by vested interests, who are not looking out for what is best for us as customers!

Why your business needs serverless. In this guest post for the Jefferson Frank, I break down how developers, managers as well as business stakeholders can all benefit from serverless. For developers, serverless technologies help us build more scalable and resilient systems and let us focus on customer needs. For managers, they help my team respond to business needs faster and more cost efficiently. They allow my team to have more ownership and autonomy and reduce the amount of inter-team dependencies. As a business stakeholder, they make it possible for me to understand the operational cost of each transaction. Which allows me to make informed decisions on pricing as well as where (and when) we should optimize the system to maximise return-on-investment.

CloudFormation protip: use !Sub instead of !Join. If you’re working with CloudFormation, then you should use !Join only when you need to perform string concatenation. In every other case, !Sub is a better choice.

How to include Serverless Repository apps in serverless.yml. Here’s a simple workaround to let you include Serverless Repository apps when you’re using the Serverless framework.

AWS SAM + CloudFormation macros, a patch made in heaven. If you are using AWS SAM and find yourself constrained by the lack of a plugin system (like the Serverless framework), then you can customize/extend SAM using CloudFormation macros.

Lambda and Kinesis, beware of hot streams. In this guest post for Lumigo, I examined whether or not the scalability issues with having too many Lambda subscribers to a single Kinesis stream has been resolved with enhanced fan-out. Compared to 2017, things have improved a lot but a series issue still remains. After about 15 subscriber functions, it becomes almost impossible to add new subscribers because the deployment itself fails consistently. To wrap up the post, we also considered some workarounds for this issue.

AWS announcement

There were some big announcements from AWS regarding Lambda.

Node 10.x

Node 10 is now supported on Lambda. But there are a couple of things to note:

• It’s 10.x, so it’s pinned to the major version only. Which means AWS can update the minor versions at a later date without you having to make code changes. This should be a good thing, as we don’t miss out on new features from minor updates. But potentially it can also introduce bugs/subtle behaviour changes when the platform updates the minor version.
• The 10.x runtime runs on a brand new OS – Amazon Linux 2 (which was announced at re:invent 2018). The new base OS image is VERY LEAN – 800MB smaller than the existing base image (Amazon Linux 1). So a lot of the libs and binaries are not there anymore. If you call out to ps, tar, etc. or depend on libraries like bzip2, libcurl, etc. then you’ll need to pack those yourself going forward. If your function is pure JS then you should be fine. You can use tools like docker-lambda or SAM or Serverless framework (which supports invoke local with docker since v1.41.0) to check that your function still runs on the new OS image.
• If you are using extractedFields from CloudWatch Logs events for log parsing then Node 10.x prepends “INFO” or “ERROR” to the extracted event. I’d have liked it if it was introduced as a separate field, but it’s helpful to distinguish between stdout and stderr outputs. Specifically, when your function errors with an unhandled exception. It’s useful to get the extra context (that the message is for an error) when you’re parsing these log events.

• There is an open bug that function doesn’t start when the event loop is not empty. More details here.

For more details about Node 10.x, read this good thread by Michael Hart from Bustle. Despite all the excitement, I think you should hold off upgrading to the Node 10.x runtime just yet. At least until some of these early bugs/issues are ironed out.

Updates to all existing functions

AWS is rolling out an update to the underlying OS (Amazon Linux 1) to all non-Node 10.x Lambda functions over the next month or so. This shouldn’t affect most functions, unless you depend on native bins that are compiled against the specific OS version. More details, including a recommended migration plan, is available here.

AppSync supports multiple auth methods

You can now specify additional auth modes for different GraphQL actions. So now you can use a combination of:

• IAM auth for mutation actions, so they can be only triggered by your applications.
• API keys for queries so that client apps can access the GraphQL endpoint without having to go through Cognito/IAM.