admin Yan Cui

All articles by admin

 

Many-faced threats to Serverless security

Tweet Threats to the security of our serverless applications take many forms, some are the same old foes we have faced before; some are new; and some have taken on new forms in the serverless world. As we adopt the serverless paradigm for building cloud-hosted applications, we delegate even more of the operational responsibilities to...
Continue reading...

 

The problems with DynamoDB Auto Scaling and how it might be improved

Tweet TL;DR – AWS announced the long awaited auto scaling capability for DynamoDB, but we found it takes too long to scale up and doesn’t scale up aggressively enough as it’s held back by using consumed capacity as scaling metric rather than actual request count. Here at Space Ape Games we developed an in-house tech to auto scale...
Continue reading...

 

Running and debugging AWS Lambda functions locally with the Serverless framework and VS Code

Tweet One of the complaints developers often have for AWS Lambda is the inability to run and debug functions locally. For Node.js at least, the Serverless framework and VS Code provides a good solution for doing just that. An often underused feature of the Serverless framework is the invoke local command, which runs your code locally...
Continue reading...

 

I’m running a live course on designing serverless architecture with AWS Lambda

Tweet Hi everyone, just a quick note to let you know that I’m running a live online course with O’Reilly on designing serverless architectures with AWS Lambda. It’s a 2-day course on September 11-12th with 6 hours in total, and it’s available for free if you have a subscription with SafariBooksOnline. Registration for the course is open till...
Continue reading...

 

Slides for my serverless security talk

Tweet Security in serverless world from Yan Cui Tweet...
Continue reading...

 

Applying the Saga pattern with AWS Lambda and Step Functions

Tweet The Saga pattern is a pattern for managing failures, where each action has a compensating action for rollback. In Hector Garcia-Molina’s 1987 paper, it is described as an approach to handling system failures in a long-running transactions. It has become increasingly relevant in the world of microservices as application logic often needs to transact...
Continue reading...

 

Yubl’s road to Serverless – Part 5 – building better recommendations with Lambda, BigQuery and GrapheneDB

Tweet Note: see here for the rest of the series.   When I joined Yubl in April 2016, it had launched just 2 months earlier, after a long and chaotic development cycle that lasted more than 2 years – all the while there was a fully armed sales team before there was even a product!...
Continue reading...

 

AWS X-Ray and Lambda : the good, the bad and the ugly

Tweet AWS announced general availability of AWS Lambda support for AWS X-Ray back in May. It’s taken me a while to try it out, and whilst I see plenty of values I think its current limitations significantly restricts its usefulness in a complex system. I found Lambda-specific documentations to be fragmented and I had to...
Continue reading...

 

Finding coldstarts : how long does AWS Lambda keep your idle functions around?

Tweet In the last post I compared the coldstart time for Lambda functions with different language, memory and code size. One of the things I learnt was that idle functions are no longer terminated after 5 minutes of inactivity. AWS Lambda – compare coldstart time with different languages, memory and code sizes It is a fantastic...
Continue reading...

 

AWS Lambda – compare coldstart time with different languages, memory and code sizes

Tweet A while back we looked at the performance difference between the language runtimes AWS Lambda supports natively. AWS Lambda – comparing platform performances We intentionally omitted coldstart time from that experiment as we were interested in performance differences when a function is “warm”. However, coldstart is still an important performance consideration, so let’s take...
Continue reading...

 

Beware of dilution of DynamoDB throughput due to excessive scaling

Tweet TL;DR – The no. of partitions in a DynamoDB table goes up in response to increased load or storage size, but it never come back down, ever. DynamoDB is pretty great, but as I have seen this particular problem at 3 different companies – Gamesys, JUST EAT, and now Space Ape Games – I think...
Continue reading...

 

From F# to Scala – implicits

Tweet Note: read the whole series here.   Having looked at case class and extractors recently, the next logical thing would be partial functions. Since Andrea pointed me to a really well article on the subject I don’t think there’s anything else for me to add, so instead, let’s look at Scala’s implicits, which is a very powerful...
Continue reading...

 

Serverless 1.X – enable API Gateway caching on request parameters

Tweet Having previously blogged about the untrodden path to enable caching on API Gateway request parameters in the Serverless framework 0.5.X, it’s a little disappointing that it’s still not officially fixed in the 1.X versions… The Problem The problem is two-fold: there’s currently no way to specify caching should be enabled for path & query...
Continue reading...

 

Yubl’s road to Serverless architecture – Part 4 – building a scalable push notification system

Tweet The Road So Far part 1 : overview part 2 : testing and continuous delivery strategies part 3 : ops   Just before Yubl’s untimely demise we did an interesting piece of work to redesign the system for sending targeted push notifications to our users to improve retention. The old system relied on MixPanel for both selecting users as...
Continue reading...

 

Auto-scaling Kinesis streams with AWS Lambda

Tweet Following on from the last post where we discussed 3 useful tips for working effectively with Lambda and Kinesis, let’s look at how you can use Lambda to help you auto scale Kinesis streams. Auto-scaling for DynamoDB and Kinesis are two of the most frequently requested features for AWS, as I write this post I’m sure the...
Continue reading...

 

AWS Lambda —3 pro tips for working with Kinesis streams

Tweet At Yubl, we arrived at a non-trivial serverless architecture where Lambda and Kinesis became a prominent feature of this architecture. Whilst our experience using Lambda with Kinesis was great in general, there was a couple of lessons that we had to learn along the way. Here are 3 useful tips to help you avoid...
Continue reading...

 

AWS Lambda – build yourself a URL shortener in 2 hours

Tweet An interesting requirement came up at work this week where we discussed potentially having to run our own URL Shortener because the Universal Links mechanism (in iOS 9 and above) requires a JSON manifest at https://domain.com/apple-app-site-association Since the OS doesn’t follow redirects this manifest has to be hosted on the URL shortener’s root domain. Owing to a limitation...
Continue reading...

 

Slides for my AWS user group talk – AWS Lambda from the trenches

Tweet Hello, just a quick note to say thanks to everyone who attended the AWS user group meetup last night, was a pleasure to talk to many of you afterwards. As promised, here are the slides for my talk from last night. AWS Lambda from the trenches from Yan Cui Tweet...
Continue reading...

 

AWS Lambda – comparing platform performances

Tweet As Lambda adds nodejs 6.10 to its supported platforms I wondered if there’s any performance differences between the platforms. Thankfully the templates in the Serverless framework make it a relative breeze to test it out with a simple HelloWorld function.   The Test see the test code here. I created a simple Lambda function for each platform that...
Continue reading...

 

Yubl’s road to Serverless architecture – Part 3 – ops

Tweet Note: see here for the rest of the series.   A couple of folks asked me about our strategy for monitoring, logging, etc. after part 2, and having watched Chris Swan talk about “Serverless Operations is not a Solved Problem” at the Serverless meetup it’s a good time for us to talk about how we approached ops with...
Continue reading...

 

Yubl’s road to Serverless architecture – Part 2 – Testing and CI/CD

Tweet Note: see here for the rest of the series.   Having spoken to quite a few people about using AWS Lambda in production, testing and CI/CD are always high up the list of questions, so I’d like to use this post to discuss the approaches that we took at Yubl. Please keep in mind that this...
Continue reading...

 

From F# to Scala – apply & unapply functions

Tweet Note: read the whole series here.   Last time around we looked at Scala’s Case Class in depth and how it compares to F#’s Discriminated Unions. F# also has Active Patterns, which is a very powerful language feature in its own right. Unsurprisingly, Scala also has something similar in the shape of extractors (via the unapply...
Continue reading...

 

From F# to Scala – case class/object (ADTs)

Tweet Note: read the whole series here.   Continuing on from where we left off with traits last time around, let’s look at Scala’s case class/object which can be used to create Algebraic Data Types (ADTs) in Scala.   Case Class You can declare an ADT in F# using Discriminated Unions (DUs). For example, a binary tree might be represented...
Continue reading...

 

From F# to Scala – traits

Tweet Note: read the whole series here.   Continuing on from where we left off with type inference last time around, let’s look at a language feature in Scala that doesn’t exist in F# – traits. Scala has both abstract classes and traits (think of them as interfaces, but we’ll get into the differences shortly) to support...
Continue reading...

 

From F# to Scala – type inference

Tweet It’s a new year, a new job, a new language (Scala) and runtime (JVM) to work with. After a week of learning about my new surroundings at Space Ape Games and getting up to speed with Scala I have learnt a fair bit. In my own journey (aided by Twitter’s Scala School, Odersky’s Coursera course and...
Continue reading...

 

Year in Review, 2016

Tweet Changes, Changes, Changes On a personal front, 2016 has been a year of great highs and lows. I left JUST EAT in March, and started working at Yubl where I had the pleasure of working with an amazing group of people and had lots of fun building things with AWS Lambda and Node.js. We took...
Continue reading...

 

Advent of Code F# – Day 25

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 25 See details of the challenge here. Today’s input looks like this: cpy a d cpy 7 c cpy 362 b inc d dec b jnz b -2 … At last, we’re at the end of this year’s Advent...
Continue reading...

 

Advent of Code F# – Day 24

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 24 See details of the challenge here. Today’s challenge is a mix of Breadth-First Search (or Level Order Tree Traversal) and the Travelling Salesman Problem. First, let’s parse the input file into a 2D array to make it easier...
Continue reading...

 

Advent of Code F# – Day 23

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 23 See details of the challenge here. Today’s input looks like this: cpy a b dec b cpy a d cpy 0 a cpy b c inc a .. Today’s challenge is an extension to Day 12, where we...
Continue reading...

 

Advent of Code F# – Day 22

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 22 See details of the challenge here. Today’s input looks like this: root@ebhq-gridcenter# df -h Filesystem                           Size    Used     Avail    Use% /dev/grid/node-x0-y0  ...
Continue reading...

 

Advent of Code F# – Day 21

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 21 See details of the challenge here. Today’s input looks like this: swap position 2 with position 7 swap letter f with letter a swap letter c with letter a rotate based on position of letter g rotate based...
Continue reading...

 

Advent of Code F# – Day 20

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 20 See details of the challenge here. Today’s input looks like this: 1873924193-1879728099 2042754084-2076891856 4112318198-4113899685 1039794493-1057170966 3791841434-3797494664 1518668516-1518748952 1946127596-1953926346 4058215215-4086224696 3429681642-3455096313 2599576643-2604275147 1800210010-1801990849 1761160149-1766904471 2774395403-2774748831 1520470679-1542287000 2343327790-2346083217 … First, let’s read the input into a list of tuples representing...
Continue reading...

 

Advent of Code F# – Day 19

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 19 See details of the challenge here. I initially approached today’s challenge as a dynamic programming exercise, but it quickly transpired that there’s a much better way to do it once I realised that part 1 is in fact...
Continue reading...

 

Advent of Code F# – Day 18

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 18 See details of the challenge here. To solve both part 1 and 2:   Links Day 18 challenge description Advent of Code 2015 Solution for Day 17 All my F# solutions for Advent of Code Github repo Tweet...
Continue reading...

 

Advent of Code F# – Day 17

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 17 See details of the challenge here. First, let’s add a hash function (that returns the MD5 as a hexadecimal string as we have done so often this year). Then, add a step function that’ll take a (x, y) position and the...
Continue reading...

 

Advent of Code F# – Day 16

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 16 See details of the challenge here. I also tried an implememtation using string instead of bool[] and there is no distinguishable difference in terms of performance. Now that we have the solve function we can answer both part...
Continue reading...

 

Advent of Code F# – Day 15

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 15 See details of the challenge here. Today’s challenge is pretty straight forward, and a brute force approach would have suffice – try every t from 0 to inifinity and return the first t that satisfies this equation for...
Continue reading...

 

Advent of Code F# – Day 14

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 14 See details of the challenge here. Today’s challenge is very similar to that of Day 5, but the requirements for a valid hash value is different this time. As before, we’ll start by defining a hash function that will accept a string and...
Continue reading...

 

Advent of Code F# – Day 13

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 13 See details of the challenge here. Today’s challenge involves solving two separate problems: counting set bits in an integer level order tree traversal For the first problem, I recommend reading through this blog post which covers several approaches and optimizations for...
Continue reading...

 

Equilibrium Index problem in F#

Tweet A good friend pointed me to a nice coding challenge she encountered – finding the Equilibrium Index of an array in O(n) time. After a quick discussion we ended up with 2 solutions, and naturally I had to code them up in F#   Solution 1 This is the same approach to the one you saw...
Continue reading...

 

Advent of Code F# – Day 12

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 12 See details of the challenge here. The input for today’s challenge looks like this: cpy 1 a cpy 1 b cpy 26 d jnz c 2 jnz 1 5 cpy 7 c … This is very similar to Day...
Continue reading...

 

Advent of Code F# – Day 11

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 11 See details of the challenge here. Today is perhaps the first AOC challenge that is really challenging and required a lot of thought, and I really enjoyed it! First, let’s model the problem domain. and we’ll need a way...
Continue reading...

 

Implementing a BST in F#

Tweet Binary Search Trees (BST) seem to be a hot topic in technical interviews. So, in order to help me prepare for all these grilling technical tests, I thought I’d give it a crack at implementing insertion, deletion and verification logic since I’ve already done both breath-first and depth-first traversals.   Verification Starting with something nice...
Continue reading...

 

Depth-First Tree Traversal in F#

Tweet We looked at breath-first tree traversal earlier today, now let’s take a look at depth-first tree traversal as well. Here, we loop through the tree starting with the root, yield the value for each of the nodes before recursively traversing down all the left branch first and then the right branch.   Try it Yourself   Links...
Continue reading...

 

Level Order Tree Traversal in F#

Tweet Another common problem I see during my preparation for technical interviews is the Level Order Tree Traversal (or Breadth-First) problem. I always like to think with my F# hat on when I solve algorithm problem, so here’s an implementation in F# . Here we have a nested recursive loop function that: yields the value from the...
Continue reading...

 

Advent of Code F# – Day 10

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 10 See details of the challenge here. The input for today’s challenge looks like this: bot 171 gives low to bot 4 and high to bot 84 bot 1 gives low to bot 117 and high to bot 81 bot...
Continue reading...

 

Advent of Code F# – Day 9

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 9 See details of the challenge here. The input for today’s challenge is a very long string like this: (6×9)JUORKH(10×13)LNWIKDMACM(126×14)(21×8)QLKUJNVVZIQGGFCJZMPHK(2×1)ZH(59×3)(38×14)KELEPIDYLCGJUBCXACRSOCEZYXLO… First, let’s see how we’re gonna parse this input. The approach I went with is to recursively split the...
Continue reading...

 

O(n) solution to Multiply Others problem in F#

Tweet Another tasty challenge that I ran into during my preparation for technical interviews is this seemingly simple problem from Facebook. input [2,3,1,4] output [12,8,24,6] Multiply all fields except it’s own position. Restrictions: 1. no use of division 2. complexity in O(n) The main challenge is in making the algorithm run in O(n) time. One solution...
Continue reading...

 

Anagram Substring Search problem in F#

Tweet One interesting problem I read about as I prepare for various technical interviews is the so called Anagram Substring Search problem where it can be boiled down to the question “find all permutations of s within b“. Naturally, after all my endeavours with Project Euler and Advent of Code, I cast my mind to how I’d solve...
Continue reading...

 

Advent of Code F# – Day 8

Tweet ps. look out for all my other solutions for Advent of Code challenges here.   Day 8 See details of the challenge here. The input for today’s challenge looks like this: rect 1×1 rotate row y=0 by 10 rect 1×1 rotate row y=0 by 10 rect 1×1 rotate row y=0 by 5 rotate column x=43...
Continue reading...