Slides and recording of my Lambda talk at LeetSpeak 2016

New releases – DynamoDB.SQL and Darkseid

Hi, just a quick update on two of my libraries aimed at making AWS easier to work with from .Net.

 

DynamoDB.SQL

DynamoDB.SQL is a SQL-like external DSL for querying & scanning data in Amazon DynamoDB. Version 3.0.0 has been released, which moves away from the monolithic .Net AWSSDK (v2.x.x), and onto the DynamoDB specific package.

You can continue to use v2.x packages for DynamoDB.SQL, I’ll apply any bug fixes to both v2.x and v3.x packages. However, any new features in the future – such as support for the mid-level Table abstractions in the AWSSDK – will be added to v3.x only.

Over the coming weeks and months, I’ll continue the effort to migrate my fleet of AWS-related tools and libraries to the service-specific packages.

 

Darkseid

Darkseid is a producer library for Amazon Kinesis, it works hand-in-hand with ReactoKinesix which provides the consumer side of the story.

Courtesy of Dustin’s PR, version 0.3.0 has been released which adds synchronous methods for pushing events to Kinesis (so that services that aren’t ready to go async all the way can still integrate with Kinesis using Darkseid).

 

That’s it, folks, hope you all had a nice weekend. If you haven’t seen Deadpool yet, you should, it is amazing 

Introducing log4net.Kinesis, a log4net appender for Amazon Kinesis

Just under three weeks ago, Amazon announced the public availability of their new Kinesis service, a service which is designed to allow real-time processing of streaming big data.

As an experiment I have put together a simple, actor-based customer appender for log4net which allows you to publish your log messages into a configured Kinesis stream. You can then have another cluster of machines to fetch the data from the stream and do whatever processing or aggregation you like to do.

You can download and install the appender from Nuget here or checkout the source code here.

The implementation is done in F# in 100 lines of code, and as you can see is very simple, easy to reason with, fully asynchronous and thread-safe.

 

Once you have pushed your log messages into the stream, you’ll need to use the AWSSDK to fetch the data and process them. For Java, there’s a client application which takes care of most of the heavy lifting – e.g. tracking your progress, handling failovers and load balancing. Unfortunately, at the time of writing, there’s no equivalent of such client application in the current version of the .Net AWSSDK.

So to help make it easier for us .Net folks to build real-time data processing applications on top of Amazon Kinesis, I had started a Rx-based .Net client library called ReactoKinesiX (I really wanted to get RX into the name!), more details to follow.

 

I think the introduction of Kinesis is very exciting and opens up many possibilities, and at the current pricing model it also represents a very cost effective alternative to some of the other competing and more polished services out there.