The Serverless framework lets you reference external JSON, YML and JS files using the syntax ${file:(fileName):propertyName}
. However, you can’t customize these external config files with runtime arguments.
A reader asked me:
“I have this boilerplate that is repeated many times in my serverless.yml, how do I reuse the boilerplate but override only specific fields?”
It’s an interesting question and here are some potential solutions.
Disclaimer: the below examples use the HTTP event source for the sole purpose of illustrating the solutions with a trivial configuration object.
YML anchors
The simplest solution is to use YML anchors. You can define and apply the anchor and override specific fields. For example:
If the boilerplate is large, then you might wish to put the boilerplate into a separate file. Unfortunately, I haven’t found a way to make anchors work with external config files in the serverless.yml
. I’m not able to import the content of an anchor from another file (as below). Please let me know in the comments if you know what I have done wrong here.
A bigger problem with anchors is that it’s difficult to replace nested fields. For example, given the following anchor:
defaultPerson: &defaultPerson location: city: Amsterdam country: Netherlands occupation: title: independent consultant expertise: - serverless - aws
Suppose I want to change location.city
and insert a new field occupation.company
. Look what happens if I apply the overrides before and after the anchor:
AFAIK, the only way to make this work is to declare multiple anchors:
defaultPerson: &defaultPerson location: &defaultLocation city: Amsterdam country: Netherlands occupation: &defaultOccupation title: independent consultant expertise: - serverless - aws Yan: <<: *defaultPerson location: <<: *defaultLocation city: Amstelveen occupation: <<: *defaultOccupation company: theburningmonk ltd
It’s getting a bit messy though.
JS Proxy objects
The Serverless framework lets you reference an external JS file too. But you can only reference properties and not invoke methods. This makes it difficult to make a customizable boilerplate config.
For example, if I want to template the HTTP event source configuration but allow the method to be customizable. Ideally, I would like to be able to do something like this:
index: handler: index.handler events: - http: ${file(defaultHttp.js):withMethod("get")} - http: ${file(defaultHttp.js):withMethod("post")}
Unfortunately, since I can’t invoke methods, I would need to wrap them into properties, e.g.
const generateConfig = (method) => ({ path: '/', method }); module.exports.get = () => generateConfig('get'); module.exports.post = () => generateConfig('post');
Then I would be able to reference them in the serverless.yml
:
index: handler: index.handler events: - http: ${file(defaultHttp.js):get} - http: ${file(defaultHttp.js):post}
That’s a two-step process. Fortunately, with ES6 Proxy we can do better!
We can return a Proxy
that traps any attempt to access a property on the exported object. The attempted property name (e.g. get
) is used to construct the actual config object we will return.
const handler = { get: function (obj, method) { return () => ({ path: '/', method }); } } module.exports = new Proxy({}, handler);
And this is what happens when I reference ${file(xyz.js):get}
:
With this simple technique, I can create a boilerplate config which you can customize a field even if it’s deeply nested. For example:
const handler = { get: function (obj, city) { return () => ({ location: { city, // customize a nested field country: 'Netherlands' }, occupation: { title: "independent consultant", expertise: ["serverless", "aws"] } }); } } module.exports = new Proxy({}, handler);
And import the config like this:
custom: yan: ${file(defaultPerson.js):Amstelveen}
But wait! Can I insert/update a property at an arbitrary location, like in the defaultPerson
example earlier?
Well, yes, you can do that by returning another Proxy
from the first Proxy
.
For example, the following config object lets you update either path
or method
, or insert a new field.
const _ = require('lodash'); const template = { path: '/', method: 'get' }; const handler = { get: function (obj, path) { const x = _.cloneDeep(obj); return () => new Proxy(x, { get: function (obj, value) { _.set(obj, path, value); return obj; } }); } } module.exports = new Proxy(template, handler);
You can then reference it in the serverless.yml
like this:
functions: index: handler: index.handler events: - http: ${file(defaultHttp.js):path./index} - http: ${file(defaultHttp.js):method.post}
Which creates the endpoints as you’d expect:
endpoints: GET - https://xxx.execute-api.us-east-1.amazonaws.com/dev/index POST - https://xxx.execute-api.us-east-1.amazonaws.com/dev/
This is what happens when I reference ${file(xyz.js}:method.post}
:
But wait again! Can you customize the config with more than one variable? E.g. what if I want to customize both path
and method
?
You can have a Proxy
that returns another Proxy
that returns another Proxy
and so on. So conceivably it’s possible.
As suggested by Philipp Beau, you can also create a scheme to support key-value pairs in a comma-separated fashion. For example, ${file(xyz.js):key0_value0-key1_value1}
I’ll leave that as an exercise to you, the readers. Let your imaginations go wild and let me know what practical problems you are trying to solve. (I love a good hack, but solving actual problems are always more rewarding ;-))
Hi, I’m Yan. I’m an AWS Serverless Hero and the author of Production-Ready Serverless.
I specialise in rapidly transitioning teams to serverless and building production-ready services on AWS.
Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.
Check out my new course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. Including basic concepts, HTTP and event triggers, activities, callbacks, nested workflows, design patterns and best practices.
Come learn about operational BEST PRACTICES for AWS Lambda: CI/CD, testing & debugging functions locally, logging, monitoring, distributed tracing, canary deployments, config management, authentication & authorization, VPC, security, error handling, and more.
You can also get 40% off the face price with the code ytcui.
Further reading
Here is a complete list of all my posts on serverless and AWS Lambda. In the meantime, here are a few of my most popular blog posts.
- Lambda optimization tip – enable HTTP keep-alive
- You are thinking about serverless costs all wrong
- Many faced threats to Serverless security
- We can do better than percentile latencies
- I’m afraid you’re thinking about AWS Lambda cold starts all wrong
- Yubl’s road to Serverless
- AWS Lambda – should you have few monolithic functions or many single-purposed functions?
- AWS Lambda – compare coldstart time with different languages, memory and code sizes
- Guys, we’re doing pagination wrong