ICYMI: five updates you’ve missed about Serverless Step Functions

You can become a serverless blackbelt. Enrol to my 4-week online workshop Production-Ready Serverless and gain hands-on experience building something from scratch using serverless technologies. At the end of the workshop, you should have a broader view of the challenges you will face as your serverless architecture matures and expands. You should also have a firm grasp on when serverless is a good fit for your system as well as common pitfalls you need to avoid. Sign up now and get 15% discount with the code yanprs15!

Over the last 6 months, we have made the Serverless Step Functions plugin better and more useful. Here are the most impactful changes, in case you missed it!

Support for intrinsic function

One of the main pain points of using the plugin has long been that you needed to use fully-formed ARNs. As of v1.18.0 you can use CloudFormation intrinsic functions Fn:GetAtt and Ref for Task states.

This works for both Task states with Lambda functions as well as non-Lambda services such as SNS, SQS, DynamoDB, ECS and Batch.

      name: myStateMachine
        StartAt: HelloWorld1
            Type: Task
              Fn::GetAtt: [HelloLambdaFunction, Arn]
            End: true

Integration with non-Lambda services

Speaking of non-Lambda services. Step Functions announced support for eight more services (besides Lambda) at re:invent 2018. This has been supported in the serverless-step-functions plugin as of v1.10.0.

SNS and SQS examples are available in this example repo. They are also covered in my video course Complete guide to AWS Step Functions.

To publish a message to SNS, you need something like the following:

      name: myStateMachine
        StartAt: Publish SNS message
          Publish SNS message:
            Type: Task
            Resource: arn:aws:states:::sns:publish
              Message: "{ \"answer\": 42 }"
                Ref: AlarmTopic
            End: true

When you integrate with these non-Lambda services, the plugin would auto-generate the corresponding IAM permissions for the execution role.

At the moment, SNS, SQS, DynamoDB, ECS and Batch are supported.

CloudWatch Alarms

It’s common practice to want to monitor the health of your workflows and be alerted when something goes wrong. You can now use the built-in alarms configuration to generate a set of default alarms for each state machine.

          ok: arn:aws:sns:...
          alarm: arn:aws:sns:...
          insufficientData: arn:aws:sns:...
          - executionsTimeOut
          - executionsFailed
          - executionsAborted
          - executionThrottled
        treatMissingData: missing

The generated CloudWatch alarms would have the following configurations:

namespace: 'AWS/States'
metric: <ExecutionsTimeOut | ExecutionsFailed | ExecutionsAborted | ExecutionThrottled>
threshold: 1
period: 60
evaluationPeriods: 1
ComparisonOperator: GreaterThanOrEqualToThreshold
Statistic: Sum
treatMissingData: <missing (default) | ignore | breaching | notBreaching>
  - Name: StateMachineArn
    Value: <ArnOfTheStateMachine>

Workflow execution events

Step Functions recently announced support for Workflow execution events. With this new feature, you can set up automated notifications when the status of a workflow starts/completes/errors through CloudWatch Events. Events can be delivered to AWS Lambda, SNS, SQS, Kinesis, or AWS Step Functions for automated response to the event.

With the serverless-step-functions plugin, you can set up notifications when a workflow’s status changes to ABORTED, FAILED, RUNNING, SUCCEEDED or TIMED_OUT. Here’s how you can configuration notifications:

      name: test
          - sns: SNS_TOPIC_ARN
          - sqs: SQS_TOPIC_ARN
          - sqs: # for FIFO queues, which requires you to configure the message group ID
              arn: SQS_TOPIC_ARN
              messageGroupId: 12345
          - lambda: LAMBDA_FUNCTION_ARN
          - kinesis: KINESIS_STREAM_ARN
          - kinesis:
               arn: KINESIS_STREAM_ARN
               partitionKeyPath: $.id # used to choose the parition key from payload
          - firehose: FIREHOSE_STREAM_ARN
          - stepFunctions: STATE_MACHINE_ARN
          ... # same as above
        ... # other status

You can also use CloudFormation intrinsic functions Fn::GetAtt and Ref to reference additional resources in the serverless.yml.

LAMBDA_PROXY request template

When you use the HTTP event trigger, the default API Gateway request template only forwards the request body as input.

This means you lose out a lot of the context around the HTTP request such as HTTP headers as well as path and query string parameters. To access this information you’d have to create a custom request template.

Also, it presents a stark difference to the event payload we have come to expect when building APIs with API Gateway and Lambda.

So we updated the HTTP event trigger and gave you the option to choose the LAMBDA_PROXY request template instead.

        - http:
            path: posts/create
            method: POST
              template: lambda_proxy

This gives you a close approximation to the event payload you can expect from API Gateway when you use LAMBDA_PROXY integration method. However, multi-value query string parameters do not work at the moment. This owes to the fact that API Gateway does not pass them through to the integration request. Once that is addressed we will add support for it.

Other changes

So those are the five most noteworthy changes in the last 6 months. To wrap things up, here are a couple of smaller changes that you might also be interested in:

If you’re interested in learning more about Step Functions and where it fits into your wider solution. Please check out my course – Complete guide to AWS Step Functions.

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my newsletter

Hi, I’m Yan. I’m an AWS Serverless Hero and I help companies go faster for less by adopting serverless technologies successfully.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.

Skill up your serverless game with this hands-on workshop.

My 4-week Production-Ready Serverless online workshop is back!

This course takes you through building a production-ready serverless web application from testing, deployment, security, all the way through to observability. The motivation for this course is to give you hands-on experience building something with serverless technologies while giving you a broader view of the challenges you will face as the architecture matures and expands.

We will start at the basics and give you a firm introduction to Lambda and all the relevant concepts and service features (including the latest announcements in 2020). And then gradually ramping up and cover a wide array of topics such as API security, testing strategies, CI/CD, secret management, and operational best practices for monitoring and troubleshooting.

If you enrol now you can also get 15% OFF with the promo code “yanprs15”.

Enrol now and SAVE 15%.

Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!