Check out my new course Learn you some Lambda best practice for great good! and learn the best practices for performance, cost, security, resilience, observability and scalability.
I’m starting a new job at DAZN next week, exciting times! I’m looking forward to working in a very different domain, after many wonderful years in the games industry. Also, I’ll be looking for talented engineers to join my team. If you’re interested, then take a look at the job spec to get a sense of what the role involves.
In the spirit of trying something new I’m trialing a new format of blog posts in the shape of weekly updates. In these updates I will share interesting articles & announcements I come across, as well as a summary of what I have been working out or writing about.
Welcome TotalCloud as our first ever sponsor!
First, I wanna give a shout out to TotalCloud who will be sponsoring this blog for the next week.
TotalCloud, the first-of-its-kind interactive & immersive visual console for cloud management, provides DevOps engineers and IT managers with real-time, virtual view of cloud inventory, superimposed with additional layers of contextual insights and operational capabilities, ultimately to provide “complete cloud visibility.”
Using visually-rich 3D representation of metric data and real-time strategy (RTS) gamification concepts, it allows users to perform cost analysis, manage, optimize, comply, & secure cloud resources from a single touch point — visually, contextually and interactively.
I love sharing, but maintaining this blog and writing regularly takes a lot of time and effort. Not to mention there is a cost for running the blog itself as well, hence all the ads you normally see on here. My goal with sponsorship is to work with sponsors who are aligned with the interest of myself and my readers, and find a sustainable way to keep this blog going without relying on ads and improve the viewing experience for everyone.
If you’re interested in sponsoring this blog in the future, please check out the sponsorship page for more details.
AWS Lambda supports Node.js 8.10
The biggest announcement in the serverless space for me is that Node.js 8.10 is finally added to AWS Lambda. You can now use the async and await keywords with Promise, and no longer need to use co as polyfill! Here are some other things that you get with Node.js 8 support.
AWS Summit SF
In typical AWS fashion, they announced a whole bunch of things during the AWS Summit in SF this week. Here are a few that I picked up on:
- A new AWS Secrets Manager service for storing secrets, it supports auto-rolling, KMS encryption and you can specify your own Lambda functions to rotate the secret. It looks interesting, but at the moment I’m not sure when I would use it instead of SSM parameter stores.
- SQS event source for Lambda is coming! I’ve been waiting for this for a long while, finally.
- S3 Select, AWS Transcribe and AWS Translate are all generally available. A lot of data folks who work with S3 and Athena would be very happy about S3 Select going GA.
- AWS Certificate Manager (ACM) launched a new Private Certificate Authority feature.
- You can use AWS Config Rules to aggregate compliance data across multiple regions and accounts
Lastly, AWS SAM (AWS’s own deployment framework based on CloudFormation) is now open sourced.
I will be hosting the serverless track at QCon New York in June
I’m pleased to announce that I will be hosting the serverless track at the QCon New York conference at the end of June! It’s my first time hosting a track at a big conference like QCon, and it’ll be my first time visiting the big apple too :-)
(new talk) Applying principles of chaos engineering to serverless
I gave a new talk at CityJsConf last week, based on my (not yet complete) series on how we can apply principles of chaos engineering to the serverless paradigm. It went down really well with the audience and I’m looking forward to giving this talk again in the future.
I’m speaking at WEBdeLDN on 10.04.2018
Next Tuesday, I will be speaking at an event organized by WEBdeLDN on “less is more”, get your ticket here. I will share my experience running serverless in production, and I will be joined by my old buddy Scott Smethurst who I worked closely with at Yubl.
New posts this week
7 things I learnt from one of the best startups you’ll find. As I take my leave from Space Ape Games, I wrote up the 7 things that I learnt from my time there. It’s a very progressive and innovative company, and it does so well in places other more established company fail.
Here are my top tips on technical writing after 8 years and 700 posts. I have been writing on this blog for a long time now, and I have developed a certain style of writing over time. In this post I shared the 6 simple rules that I follow whenever I write.
What is the best event source for doing pub-sub with AWS Lambda? In this post I discuss the tradeoffs between SNS, Kinesis Streams and DynamoDB Streams as event source for Lambda in the context of pub-sub. The discussion centers around the tradeoffs between scalability, parallelism, resilience and cost.
Progress update on Production-Ready Serverless
Last week, unit 7 was added to the MEAP (Manning Early Access Program). It covers real-time event processing with Kinesis and Lambda, as well as a bonus module on implementing per-function IAM roles.
During my time off this week, I have been working hard on unit 8 (logging) and 9 (monitoring) to get them into the hands of my editor ASAP.
In unit 8, I walked through the steps to implement log aggregation for Lambda using Logz.io, which offers hosted ELK stack and a free tier. I discussed the need for using structured logging with JSON, and how to capture additional context to enrich our logs. I discussed why you shouldn’t enable debug logging in production, and why you should instead sample debug logs in production. I then integrated the demo app with middy to make it easy to sample debug logs at a configurable sample rate.
In unit 9, I discussed the challenges the serverless paradigm poses to existing tools and practices we use for monitoring. I demonstrated how you can record custom metrics both synchronously as well as asynchronously (via CloudWatch logs). I also showed a handy trick for getting the memory usage and actual billed milliseconds out of CloudWatch logs and record them as metrics. Not stopping there, I finished the unit by demonstrating an automation pattern, to auto-create CloudWatch alarms whenever you create a new API.
Some good reads from this week
Jeremy Daly wrote a nice post on couple of things to help you manage RDS connections better from Lambda.
Tom McLaughlin mentioned on Slack about this new IDE called Sigma for writing Lambda functions.
Here’s a nice post that compares DNS Resolver Performance between CloudFlare, Google, Quad9 and OpenDNS.
Chris Munns wrote about how to manage cross-account serverless microservices.
And this is the best login page design I have ever seen!
— Darin Senneff (@dsenneff) March 28, 2018
Until next week..
So that’s it for the first edition of my weekly updates! As it’s a new format, I’d love to know what you think, and if there’s anything specific you’d like to see in these updates. Please let me know via comments or the contact form.
See you next week!
- Production-ready Serverless (get 40% off with discount code ytcui)
- What is the best event source for doing pub-sub with AWS Lambda?
- My top tips on technical writing after 8 years and 700 posts
- 7 things I learnt from one of the best startups you’ll find
- Node.js 8.10 runtime now available in AWS Lambda
- What Node.js 8 support mean for serverless developers
- AWS Secrets Manager announcement
- S3 Select is GA
- AWS Transcribe is GA
- AWS Translate is GA
- ACM launches Private Certificate Authority
- AWS Config Rules can now aggregate compliance data across regions and accounts
- How to manage RDS connections from AWS Lambda
- How to manage cross-account Serverless microservices
I specialise in rapidly transitioning teams to serverless and building production-ready services on AWS.
Are you struggling with serverless or need guidance on best practices? Do you want someone to review your architecture and help you avoid costly mistakes down the line? Whatever the case, I’m here to help.
Check out my new podcast Real-World Serverless where I talk with engineers who are building amazing things with serverless technologies and discuss the real-world use cases and challenges they face. If you’re interested in what people are actually doing with serverless and what it’s really like to be working with serverless day-to-day, then this is the podcast for you.
Check out my new course, Learn you some Lambda best practice for great good! In this course, you will learn best practices for working with AWS Lambda in terms of performance, cost, security, scalability, resilience and observability. We will also cover latest features from re:Invent 2019 such as Provisioned Concurrency and Lambda Destinations. Enrol now and start learning!
Check out my video course, Complete Guide to AWS Step Functions. In this course, we’ll cover everything you need to know to use AWS Step Functions service effectively. There is something for everyone from beginners to more advanced users looking for design patterns and best practices. Enrol now and start learning!
Are you working with Serverless and looking for expert training to level-up your skills? Or are you looking for a solid foundation to start from? Look no further, register for my Production-Ready Serverless workshop to learn how to build production-grade Serverless applications!
Here is a complete list of all my posts on serverless and AWS Lambda. In the meantime, here are a few of my most popular blog posts.
- Lambda optimization tip – enable HTTP keep-alive
- You are wrong about serverless and vendor lock-in
- You are thinking about serverless costs all wrong
- Just how expensive is the full AWS SDK?
- Many faced threats to Serverless security
- We can do better than percentile latencies
- Yubl’s road to Serverless
- AWS Lambda – should you have few monolithic functions or many single-purposed functions?
- AWS Lambda – compare coldstart time with different languages, memory and code sizes
- Guys, we’re doing pagination wrong
- Top 10 Serverless framework best practices