You need to use structured logging with AWS Lambda

You can become a serverless blackbelt. Enrol to my 4-week online workshop Production-Ready Serverless and gain hands-on experience building something from scratch using serverless technologies. At the end of the workshop, you should have a broader view of the challenges you will face as your serverless architecture matures and expands. You should also have a firm grasp on when serverless is a good fit for your system as well as common pitfalls you need to avoid. Sign up now and get 15% discount with the code yanprs15!

This is a lesson that I wished I learnt when I first started using AWS Lambda in anger, it would have made my life simpler right from the start. 

But, we did get there before long, and it allowed us to track and include correlation IDs in our log messages (which are then pushed to an ELK stack)which would also include other useful information such as:

  • age of the function execution
  • whether invocation was a cold start
  • log level
  • etc.

Looking back, I fell into the common trap of forgetting the practices that had served us well in a different paradigm of developing software, at least until I figured out how to adopt these old practices for the new serverless paradigm.

In this case, we know structured logs are important and why we need them, and also how to do it well. But like many others, we started with console.log because it was simple and it worked (to a limited degree).

But, this approach has a really low ceiling:

  • you can’t add contextual information with the log message, at least not in a way that’s consistent (you may change the message you log, but the contextual information should always be there, e.g. user-id, request-id, etc.) and easy to extract for an automated process
  • as a result, it’s also hard to filter the log messages by specific attributes – e.g. “show me all the log messages related to this request ID”
  • it’s hard to control what level to log at (i.e. debug, info, warning, …) based on configuration – e.g. log at debug level for non-production environments, but log at info/warning level for production

Which is why, if you’re just starting your Serverless journey, then learn from my mistakes and write your logs as structured JSON from the start. Also, you should use whatever log client that you were using before – log4j, nlog, loggly, log4net, whatever it is – and configure the client to format log messages as JSON and attach as much contextual information as you can.

As I mentioned in the post on how to capture and forward collection IDs, it’s also a good idea to enable debug logging on the entire call chain for a small % of requests in production. It helps you catch pervasive bugs in your logic (that are easy to catch, but ONLY if you have the right info from the logs) that would otherwise require you to redeploy all the functions on the entire call chain to turn on debug logging…

So there, a simple and effective thing to do to massively upgrade your serverless architecture. Check out my mini-series on logging for AWS Lambda which covers log aggregationtracking correlation IDs, and some tips and tricks such as why and how to send custom metrics asynchronously.

Liked this article? Support me on Patreon and get direct help from me via a private Slack channel or 1-2-1 mentoring.
Subscribe to my newsletter

Hi, I’m Yan. I’m an AWS Serverless Hero and I help companies go faster for less by adopting serverless technologies successfully.

Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.

Hire me.

Skill up your serverless game with this hands-on workshop.

My 4-week Production-Ready Serverless online workshop is back!

This course takes you through building a production-ready serverless web application from testing, deployment, security, all the way through to observability. The motivation for this course is to give you hands-on experience building something with serverless technologies while giving you a broader view of the challenges you will face as the architecture matures and expands.

We will start at the basics and give you a firm introduction to Lambda and all the relevant concepts and service features (including the latest announcements in 2020). And then gradually ramping up and cover a wide array of topics such as API security, testing strategies, CI/CD, secret management, and operational best practices for monitoring and troubleshooting.

If you enrol now you can also get 15% OFF with the promo code “yanprs15”.

Enrol now and SAVE 15%.

Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.

Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!

Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!