CloudFormation protip: use !Sub instead of !Join

You can become a serverless blackbelt. Enrol to my 4-week online workshop Production-Ready Serverless and gain hands-on experience building something from scratch using serverless technologies. At the end of the workshop, you should have a broader view of the challenges you will face as your serverless architecture matures and expands. You should also have a firm grasp on when serverless is a good fit for your system as well as common pitfalls you need to avoid. Sign up now and get 15% discount with the code yanprs15!

Update 12/05/2019: as Moshe pointed out in the comments, Fn::Sub is not supported by the Serverless framework because it too uses the ${} syntax to support its own variables system. However, as is often the case with the Serverless framework, you can work around this issue with a plugin. Check out the serverless-cloudformation-sub-variables plugin which lets you use Fn::Sub in the serverless.yml. You just need to use #{VariableName} instead of ${VariableName}.

CloudFormation supports a number of intrinsic functions and Fn::Join (or !Join) is often used to construct parameterised names and paths.

The Serverless framework, for instance, uses it extensively. A quick look in a CloudFormation it generates I can see Fn::Join used for:

  • IAM policy names
  • IAM role names
  • IAM principals
  • API Gateway URIs
  • Resource ARNs

and many more.

But it’s not just the frameworks that are using Fn::Join heavily. They also show up in our own code all the time as well. For example, to construct the ARN for a resource, or the URI for an API Gateway endpoint.

I find these very hard to comprehend, and my protip for you today is to use Fn::Sub (or the !Sub shorthand) instead.

Many folks would use Fn::Sub when they need to reference pseudo parameters such as AWS::Region and AWS::AccountId, for example:

!Sub 'arn:aws:ec2:${AWS::Region}:${AWS::AccountId}:vpc/${vpc}'

But Fn::Sub also lets you provide your own parameters. For example:

  - 'arn:aws:s3:::${Bucket}/*'
  - { Bucket: Ref MyBucket }

The advantage over Fn::Join is that you can see the pattern of the interpolated string. Whereas with Fn::Join you’ll have to construct the pattern in your mind, which requires far more cognitive energy.

  - ''
  - - 'arn:aws:s3:::'
    - !Ref MyBucket
    - '/*'

Here are a few side-by-side comparisons to drive home the message.

Example 1: IAM role name

RoleName:  # hello-world-dev-{region}-lambdaRole
    - '-'
    - - 'hello-world'
      - 'dev'
      - !Ref 'AWS::Region'
      - 'lambdaRole'

with Fn::Sub:

  !Sub 'hello-world-dev-${AWS::Region}-lambdaRole

Example 2: API Gateway integration URI

Uri: # arn:{partition}:apigateway:{region}:.../{lambda}/invocations
    - ''
    - - 'arn:'
      - Ref: AWS::Partition
      - ':apigateway:'
      - Ref: AWS::Region
      - ':lambda:path/2015-03-31/functions/'
      - !GetAtt 'HelloLambdaFunction.Arn'
      - '/invocations'

with Fn::Sub:

    - 'arn:${AWS::Partition}:apigateway:${AWS::Region}:lambda:path/2015/03/31/functions/${Function}/invocations'
    - { Function: !GetAtt 'HelloLambdaFunction.Arn' }

Example 3: Lambda permission for API Gateway

SourceArn: # arn:{partition}:execute-api:{region}:.../*/*
    - ''
    - - 'arn:'
      - Ref: AWS::Partition
      - ':execute-api:'
      - Ref: AWS::Region
      - ':'
      - Ref: AWS::AccountId
      - ':'
      - Ref: ApiGatewayRestApi
      - '/*/*'

with Fn::Sub:

    - 'arn:${AWS::Partition}:execute-api:${AWS::Region}:${AWS::AccountId}:${RestApi}/*/*'
    - { RestApi: Ref: ApiGatewayRestApi }

Suffice to say that the Fn::Sub version is easier to understand in every case! Now that you have seen what Fn::Sub can do, I hope you will prefer it to Fn::Join going forward.

Finally, if you’re using the Serverless framework and need more expressive power than the intrinsic functions can offer, then check out this plugin. It lets you use a number of “extrinsic” functions such as Fn::Substring or Fn::StartsWith anywhere in your serverless.yml.

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my newsletter

Hi, I’m Yan. I’m an AWS Serverless Hero and I help companies go faster for less by adopting serverless technologies successfully.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.

Skill up your serverless game with this hands-on workshop.

My 4-week Production-Ready Serverless online workshop is back!

This course takes you through building a production-ready serverless web application from testing, deployment, security, all the way through to observability. The motivation for this course is to give you hands-on experience building something with serverless technologies while giving you a broader view of the challenges you will face as the architecture matures and expands.

We will start at the basics and give you a firm introduction to Lambda and all the relevant concepts and service features (including the latest announcements in 2020). And then gradually ramping up and cover a wide array of topics such as API security, testing strategies, CI/CD, secret management, and operational best practices for monitoring and troubleshooting.

If you enrol now you can also get 15% OFF with the promo code “yanprs15”.

Enrol now and SAVE 15%.

Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.

Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!