How to: optimize Lambda memory size during CI/CD pipeline

You can become a serverless blackbelt. Enrol to my 4-week online workshop Production-Ready Serverless and gain hands-on experience building something from scratch using serverless technologies. At the end of the workshop, you should have a broader view of the challenges you will face as your serverless architecture matures and expands. You should also have a firm grasp on when serverless is a good fit for your system as well as common pitfalls you need to avoid. Sign up now and get 15% discount with the code yanprs15!

Lambda invocations are charged in 100ms blocks of execution time. But the cost per 100ms of execution time depends on how much memory you allocate to the function. The higher the memory, the more CPU cycles and network bandwidth, but the higher the cost as well.

One of the simplest and most efficient cost optimization for Lambda is to right-size the memory allocation for your functions. It used to be a dark art that requires plenty of trial-and-error, until the aws-lambda power-tuning app from Alex Casalboni. Which uses Step Functions to execute your function against different memory sizes to help you find the best configuration for speed, cost or a balance of the two.

However, managing the deployment of the aws-lambda-power-tuning app (which is available via the Serverless Application Repository) is an extra overhead that most of us rather do without. Not to mention the headache of keeping it up-to-date whenever Alex releases new features.

So, to make power tuning functions more accessible, I added support for it in the lumigo-cli. You can tune a function with a simple command lumigo-cli powertune-lambda -r us-east-1 -n my-function --strategy balanced .

Since version 0.39.0 of the lumigo-cli, it’s now even easier to integrate it with CI/CD pipelines. Because I added support for storing the output of the app (see below) into a local file, so they can be easily parsed, and the ability to suppress the command line prompt to visualize the result.

  "power": 512,
  "cost": 8.32e-7,
  "duration": 15.411500000000002,
  "stateMachine": {
    "executionCost": 0.0003,
    "lambdaCost": 0.001075983999999999,
    "visualization": ""

Check out this repo for a demo project that automatically tunes its functions as part of its CI/CD pipeline. Let’s walk through it.

The setup

The demo project uses the Serverless framework to configure and deploy its functions. But the same approach works with other deployment frameworks too. The important thing to note is that we have three functions:

    handler: functions/cpu-intense.handler
    handler: functions/io-intense.handler
      - http:
          path: /
          method: POST
    handler: functions/mixed.handler

By default, the Serverless framework configures functions with 1024MB of memory and 6 seconds timeout.

In the project root, there is an examples folder. This is where you put the example payloads to use when tuning each of the functions.

There is also a script, where much of the magic happens during the CI/CD pipeline.

The script

As part of my projects, I usually include a script. This script encapsulates the key steps for my CI/CD pipelines, such as running tests and deploying the project. It simplifies the process of testing my CI/CD pipelines since I can easily run it locally. And it allows me to move between CI tools easily since the “meat” of the pipeline is in the scripts.

When adopted broadly, it gives you a DSL for all of your projects, e.g.

  • ./ deploy dev us-east-1 to deploy the project to the dev stage in the us-east-1 region
  • ./ int-test dev us-east-1 to run the integration tests in the dev stage, against the us-east-1 region
  • ./ e2e-test dev us-east-1 to run the end-to-end tests in the dev stage, against the us-east-1 region
  • and so on…

In this particular case, I have a command to tune the functions in this project to a strategy of my choosing – cost, speed or balanced. For example, by running ./ tune dev us-east-1 balanced .

This command can be run AFTER the functions have been deployed. As you can see, it uses the lumigo-cli to tune each of the functions.

lumigo-cli powertune-lambda \
  -r $1 \
  -n $functionName \
  -s $4 \
  -f examples/$3.json \
  -o $3-result.json \
  -z > /dev/null

Couple of things to note here:

  • the input file (-f) points to the aforementioned examples folder. The input files need to be named after the names of the functions in the serverless.yml . In this case, cpu-intense.json , io-intense.json or mixed.json.
  • the output file (-o) would be named after the same function name too.
  • the -z flag suppresses the command line prompt where the lumigo-cli asks you if you want to see a visualization of the results.

The io-intense function (see below) is an API Gateway function and extracts the url from the POST body. So, to tune this function we need to invoke it with a payload that looks like an API Gateway event.

Similar, if you have functions that process events from SNS, SQS, Kinesis and so on, you would need to tailor the payload for each. Hence the purpose of the examples folder.

Third-party tools such as Lumigo can make your life easier here. As you can see the invocation event for a function in the Transactions view. This makes it easy to capture sample payloads without having to temporarily add print statements to your functions.

If the function’s memory size doesn’t match the optimal power, then we can update it in-place using the AWS CLI.

if ((optimalPower != memorySize)); then
  echo "updating function memory size to $optimalPower..."
  aws lambda update-function-configuration \
    --region $1 \
    --function-name $functionName \
    --memory-size $optimalPower > /dev/null

update 24/03/2020: since lumigo-cli v0.40.0 you can now apply the optimal power to the function with the powertune-lambda command, without having to script it yourself.

The CI/CD pipeline

I decided to use CircleCI here because it’s free for public repos and I have used it to good effect in the past. But the approach works with other CI tools too.

Here, my entire pipeline involves setting up the dependencies and then running ./ deploy dev us-east-1 and ./ tune dev us-east-1 balanced . In practice, I’d have used a custom docker image with all the dependencies (AWS CLI, jq, etc.) pre-installed.

And when the pipeline finishes, all 3 functions’ memory size would be fine-tuned to give the best performance-to-cost ratio.

Wrap up

In this post, we saw how you can optimize the memory configuration for your functions as part of the CI/CD pipeline. This ensures that your functions are always fine-tuned to deliver the best performance at the lowest cost.

The approach is applicable to different deployment frameworks and CI tools. Broadly speaking, your pipeline would consist of these steps:

The functions’ configurations are updated in-place and the changes are never committed back into the source code. The problems with this approach are:

  • Functions need to be tuned in every environment, after every deployment.
  • The payloads might be environment-specific, which introduces extra complexity and overhead for managing these.
  • It might not be possible to tune functions that write to databases, etc. in the production environment. At the very least, you’ll have to pay special attention to manage this to avoid introducing test data to the production environment.

To address these problems, you can extend the approach I have shown you in this post. For example, instead of updating the functions in-place, you can create a PR when improvements are identified. Assuming there are no environmental factors that can drastically affect performance, then you’d only need to run tuning in the deployment pipeline for dev. This addresses the aforementioned problems.

If you can standardise on the deployment and CI tool you use, then you can automate the process even more. For example:

  • parse the serverless.yml to extract the function names so they don’t have to be hardcoded
  • auto-generate the payload based on the configured event source for each of the functions
  • create an organization-wide CLI that lets you bootstrap new projects with a templated script and CI/CD pipeline yml

And finally, here are some relevant resources for you to explore:

Happy tuning!

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my weekly newsletter

Hi, I’m Yan. I’m an AWS Serverless Hero and I help companies go faster for less by adopting serverless technologies successfully.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.

Skill up your serverless game with this hands-on workshop.

My 4-week Production-Ready Serverless online workshop is back!

This course takes you through building a production-ready serverless web application from testing, deployment, security, all the way through to observability. The motivation for this course is to give you hands-on experience building something with serverless technologies while giving you a broader view of the challenges you will face as the architecture matures and expands.

We will start at the basics and give you a firm introduction to Lambda and all the relevant concepts and service features (including the latest announcements in 2020). And then gradually ramping up and cover a wide array of topics such as API security, testing strategies, CI/CD, secret management, and operational best practices for monitoring and troubleshooting.

If you enrol now you can also get 15% OFF with the promo code “yanprs15”.

Enrol now and SAVE 15%.

Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.

Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!