Weekly update 33

Hi, welcome to another weekly update! I’ve been busy since the last time we were here, and I have lots of cool updates to share with you.

New blog posts

How to use the power of CloudFormation custom resources for great good. In this post, I described an approach for invoking a Lambda function as part of a CloudFormation deployment using a reusable custom resource. This was useful for a number of SAR apps I maintain, including the new propagate-cfn-tags SAR app below. Where I’d like to process all existing CloudFormation stacks in the region during deployment.

Step Functions course update + 50% off offer

I added lots of new content to my Step Functions course to cover all the recent updates to the service, including:

• callback patterns – a better approach for pausing workflows while you wait for an external event, no more activities and no need to run pollers.
• nested workflows – to start another workflow inside a workflow
• blue-green deployments

Lots of videos were also updated to reflect improvements to the serverless-step-functions plugin.

To celebrate the update, I’m also running a special promotion till the end of September where you can buy the course for £9.99 instead of the usual £19.99.

DAZN Lambda powertools supports Firehose + Lambda layers

The DAZN Lambda powertools now supports Kinesis Firehose as an event source. This means we are able to:

• propagate correlation IDs through Firehose events so they are automatically included in Lambda logs
• support sampling debug logs
• track the number of invocation hops on a call-chain (to help prevent infinite recursions)

You can now also deploy all 16 packages from the DAZN Lambda powertools as a single Lambda layer. You can deploy the layer as a SAR application, using this technique by Gojko Adzic.

serverless-apigateway-service-proxy adds customization for S3 proxies

The serverless-apigateway-service-proxy plugin now lets you customize the S3 proxy, which is useful in many situations. For instance, when you cannot parameter the object key using path or query string parameters. Such is the challenge that one of my clients faced, where we subscribed an S3 service proxy endpoint to a third-party webhook. The service proxy endpoint needs to be a static URL, therefore we needed more control of the request mapping to use the request ID as object key.

Kudos to Erez Rokah for implementing this awesome feature.

serverless-lumigo supports Python functions

The serverless-lumigo plugin now officially supports Python functions! If you work with Python and would like to give Lumigo a try, then it’s easier to do so than ever before with the help of this Serverless framework plugin.

[new!] propgate-cfn-tags SAR

I published another SAR app called propagate-cfn-tags. This app addresses the problem that CloudFormation tags do not propagate to some resources such as CloudWatch log groups. By deploying this app to your account, you ensure that all existing CloudFormation stacks, as well as any future CloudFormation stacks, would propagate their tags to CloudWatch Log Groups that belong to the stacks.

To read more about the app, have a look at our announcement. If you can think of any other resources that we should support, please let us know by raising a feature request here.

async-custom-metrics SAR supports Lambda usage metrics and estimated cost metric

The async-custom-metrics SAR can now parse usage reports from Lambda logs and turn them into custom metrics:

• BilledDuration: Lambda invocations are billed in 100ms blocks, see how much time you’re actually billed for.
• MemorySize: the memory size (in MBs) of the function at the time it was invoked.
• MemoryUsed: the max amount of memory (in MBs) that was used by the invocation.
• EstimatedCost: the estimated cost for the function, calculated using the BilledDuration and MemorySize.

These custom metrics incur extra cost, so you don’t have to enable them. When you deploy the latest version of the app, you can decide whether or not to enable them.

If they are both enabled, then you will find the new metrics under the AWS/Lambda namespace.

Better DX for auto-subscribe-log-group-to-arn and auto-set-log-group-retention SAR apps

When you deploy the auto-subscribe-log-group-to-arn or auto-set-log-group-retention SAR apps, I want to process all existing log groups in the region. Previously, this was done via a cron job that runs once an hour. Which means when you first deploy the app, you won’t see any changes to existing log groups. It was confusing for users, and I wanted to give them a better developer experience.

Using the technique I outlined in this blog post, we can now invoke the relevant Lambda functions during deployment to process all existing log groups. So when the deployment is done, the log groups would have been updated. Giving you the feedback and causality that you’d expect.

[new!] cloudwatch-alarms-macro SAR

I have saved the best for last. A lot of clients have me “what alarms do I need for my serverless application?”. I wrote about CloudWatch metrics and discussed common alerts you should use. Many of these are repeatable, and to help you automate the process of creating these alarms I published another SAR app called cloudwatch-alarms-macro.

This app creates a macro in your account, which you can include in any CloudFormation stack, including AWS SAM or Serverless framework templates. You can customize the behaviour of the macro with a default configuration that controls what alarms are created by default, and what thresholds to use.

For example, as a convention, every Lambda function should have an alarm for when the error rate reaches 1% or above.

For a particular CloudFormation stack, you can also override this default configuration for particular functions. For example, you may choose to disable certain alarms or to use a different threshold.

Right now, the macro can auto-generate alarms for the following resources:

• Lambda
• SQS
• API Gateway
• Step Functions

A longer post would follow in the next few days and explain the inner-workings of this macro and how you can make the best use of it.

Announcements from the community

In case you missed it, big VPC improvements are coming to Lambda! Once rolled out to your account, you will no longer experience the 10s cold start penalty for giving Lambda functions access to your VPC. Which is a showstopper for many companies looking to adopt serverless. AWS has been teasing this for a while now, and I’m really glad to hear that it’s finally being rolled out!

I also want to give a shout out to our friends at serverlesslife.com, check out their last article on how to overcome common serverless challenges.