AWS Lambda — build yourself a URL shortener in 2 hours

An inter­est­ing require­ment came up at work this week where we dis­cussed poten­tial­ly hav­ing to run our own URL Short­en­er because the Uni­ver­sal Links mech­a­nism (in iOS 9 and above) requires a JSON man­i­fest at

Since the OS doesn’t fol­low redi­rects this man­i­fest has to be host­ed on the URL shortener’s root domain.

Owing to a lim­i­ta­tion on Apps­Fly­er it’s cur­rent­ly not able to short­en links when you have Uni­ver­sal Links con­fig­ured for your app. Whilst we can switch to anoth­er ven­dor it means more work for our (already stretched) client devs and we real­ly like Apps­Fly­er’s sup­port for attri­bu­tions.

Which brings us back to the ques­tion

should we build a URL short­en­er?”

swift­ly fol­lowed by

how hard can it be to build a scal­able URL short­en­er in 2017?”

Well, turns out it wasn’t hard at all 

Lambda FTW

For this URL short­en­er we’ll need sev­er­al things:

  1. a GET /{shortUrl} end­point that will redi­rect you to the orig­i­nal URL
  2. a POST / end­point that will accept an orig­i­nal URL and return the short­ened URL
  3. an index.html page where some­one can eas­i­ly cre­ate short URLs
  4. a GET /ap­ple-app-site-asso­ci­a­tion end­point that serves a sta­t­ic JSON response

all of which can be accom­plished with API Gate­way + Lamb­da.

Over­all, this is the project struc­ture I end­ed up with:

  • using the Server­less framework’s aws-node­js tem­plate
  • each of the above end­point have a cor­re­spond­ing han­dler func­tion
  • the index.html file is in the sta­t­ic fold­er
  • the test cas­es are writ­ten in such a way that they can be used both as inte­gra­tion as well as accep­tance tests
  • there’s a script which facil­i­tates run­ning
    • inte­gra­tion tests, eg ./ int-test {env} {region} {aws_profile}
    • accep­tance tests, eg ./ accep­tance-test {env} {region} {aws_profile}
    • deploy­ment, eg ./ deploy {env} {region} {aws_profile}

Get /apple-app-site-association endpoint

See­ing as this is a sta­t­ic JSON blob, it makes sense to pre­com­pute the HTTP response and return it every time.

POST / endpoint

For an algo­rithm to short­en URLs, you can find a very sim­ple and ele­gant solu­tion on Stack­Over­flow. All you need is an auto-incre­ment­ed ID, like the ones you nor­mal­ly get with RDBMS.

How­ev­er, I find DynamoDB a more appro­pri­ate DB choice here because:

  • it’s a man­aged ser­vice, so no infra­struc­ture for me to wor­ry about
  • OPEX over CAPEX, man!
  • I can scale reads & writes through­put elas­ti­cal­ly to match uti­liza­tion lev­el and han­dle any spikes in traf­fic

but, DynamoDB has no such con­cept as an auto-incre­ment­ed ID which the algo­rithm needs. Instead, you can use an atom­ic counter to sim­u­late an auto-incre­ment­ed ID (at the expense of an extra write-unit per request).

GET /{shortUrl} endpoint

Once we have the map­ping in a DynamoDB table, the redi­rect end­point is a sim­ple mat­ter of fetch­ing the orig­i­nal URL and return­ing it as part of the Loca­tion head­er.

Oh, and don’t for­get to return the appro­pri­ate HTTP sta­tus code, in this case a 308 Per­ma­nent Redi­rect.

GET / index page

Final­ly, for the index page, we’ll need to return some HTML instead (and a dif­fer­ent con­tent-type to go with the HTML).

I decid­ed to put the HTML file in a sta­t­ic fold­er, which is loaded and cached the first time the func­tion is invoked.

Getting ready for production

For­tu­nate­ly I have had plen­ty of prac­tice get­ting Lamb­da func­tions to pro­duc­tion readi­ness, and for this URL short­en­er we will need to:

  • con­fig­ure auto-scal­ing para­me­ters for the DynamoDB table (which we have an inter­nal sys­tem for man­ag­ing the auto-scal­ing side of things)
  • turn on caching in API Gate­way for the pro­duc­tion stage

Future Improvements

If you put in the same URL mul­ti­ple times you’ll get back dif­fer­ent short-urls, one opti­miza­tion (for stor­age and caching) would be to return the same short-url instead.

To accom­plish this, you can:

  1. add GSI to the DynamoDB table on the longUrl attribute to sup­port effi­cient reverse lookup
  2. in the short­enUrl func­tion, per­form a GET with the GSI to find exist­ing short url(s)

I think it’s bet­ter to add a GSI than to cre­ate a new table here because it avoids hav­ing “trans­ac­tions” that span across mul­ti­ple tables.

Useful Links

Like what you’re read­ing? Check out my video course Pro­duc­tion-Ready Server­less and learn the essen­tials of how to run a server­less appli­ca­tion in pro­duc­tion.

We will cov­er top­ics includ­ing:

  • authen­ti­ca­tion & autho­riza­tion with API Gate­way & Cog­ni­to
  • test­ing & run­ning func­tions local­ly
  • CI/CD
  • log aggre­ga­tion
  • mon­i­tor­ing best prac­tices
  • dis­trib­uted trac­ing with X-Ray
  • track­ing cor­re­la­tion IDs
  • per­for­mance & cost opti­miza­tion
  • error han­dling
  • con­fig man­age­ment
  • canary deploy­ment
  • VPC
  • secu­ri­ty
  • lead­ing prac­tices for Lamb­da, Kine­sis, and API Gate­way

You can also get 40% off the face price with the code ytcui. Hur­ry though, this dis­count is only avail­able while we’re in Manning’s Ear­ly Access Pro­gram (MEAP).