Recently, I have been helping a client implement an event-sourced system. In the process, I put together a very simple demo app to illustrate how one could build such a system using Lambda and DynamoDB. The source code is available on GitHub here. Before you go ahead and read all about the demo app, I …
A simple event-sourcing example with snapshots using Lambda and DynamoDB Read More »
A while back I wrote about using DynamoDB TTL to implement ad-hoc scheduling. It generated some healthy debate and a few of you have mentioned alternatives including using Step Functions. So let’s take a look at some of these alternatives, starting with the simplest – using a cron job. We will assess this approach using the …
Using CloudWatch and Lambda to implement ad-hoc scheduling Read More »
Update 15/03/2019: Thanks to Zac Charles who pointed me to this new page in the DynamoDB docs. It explains how the OnDemand capacity mode works. Turns out you DON’T need to pre-warm a table. You just need to create the table with the desired peak throughput (Provisioned), and then change it to OnDemand. After you change the table to OnDemand …
Understanding the scaling behaviour of DynamoDB OnDemand tables Read More »
CloudWatch Events let you easily create cron jobs with Lambda. However, it’s not designed for running lots of ad-hoc tasks, each to be executed once, at a specific time. The default limit on CloudWatch Events is a lowly 100 rules per region per account. It’s a soft limit, so it’s possible to request a limit …
In the last post, we look at how you can implement pub-sub with AWS Lambda. We compared several event sources you can use, SNS, Kinesis streams and DynamoDB streams, and the tradeoffs available to you. Let’s look at another messaging pattern today, push-pull, which is often referred to as fan-out/fan-in. It’s really two separate patterns …
AWS offers a wealth of options for implementing messaging patterns such as pub-sub with Lambda, let’s compare and contrast some of these options. The pub-sub pattern Publish-Subscribe (often shortened to pub-sub) is a messaging pattern where publishers and subscribers are decoupled through an intermediary broker (ZeroMQ, RabbitMQ, SNS, etc.). SNS + Lambda In the AWS ecosystem, the obvious …
What is the best event source for doing pub-sub with AWS Lambda? Read More »
“Guys, we’re doing pagination wrong…” These are the words that I had to mutter quite a few times in my career, at the dissatisfaction of how pagination had been implemented on several projects. Still, that dissatisfaction is nothing compared to how I feel when I occasionally had to ask “why is this API not paginated..?” So, …
TL;DR – AWS announced the long awaited auto scaling capability for DynamoDB, but we found it takes too long to scale up and doesn’t scale up aggressively enough as it’s held back by using consumed capacity as scaling metric rather than actual request count. Here at Space Ape Games we developed an in-house tech to auto scale DynamoDB …
The problems with DynamoDB Auto Scaling and how it might be improved Read More »
AWS announced general availability of AWS Lambda support for AWS X-Ray back in May. It’s taken me a while to try it out, and whilst I see plenty of values I think its current limitations significantly restricts its usefulness in a complex system. I found Lambda-specific documentations to be fragmented and I had to rely …
aws x-ray and lambda : the good, the bad and the ugly Read More »
update Jan 26, 2020: since this article was already written, the problem of throughput dilution has been resolved by DynamoDB’s adaptive capacity feature which is applied in real-time. TL;DR – The no. of partitions in a DynamoDB table goes up in response to increased load or storage size, but it never come back down, ever. DynamoDB …
Beware of dilution of DynamoDB throughput due to excessive scaling Read More »
By continuing to use the site, you agree to the use of cookies. more information
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.