Hi, just a quick note to say that recordings of my talk with Community for F# is now available.
In this session, I shared some of our experiences of building a successful MMORPG for a social audience and insights into some of the technical challenges that my team has had to overcome along the way.
How you enjoy them!
A couple of weeks earlier, Amazon announced support for Local Secondary Indexes (LSI) for DynamoDB. You can now perform fast, efficient queries against DynamoDB tables using attributes that are not part of the existing Hash and Range key model without resorting to the use of scans.
As a result to the new feature the DynamoDB query API has also gone through some changes, as did the AWSSDK for .Net. From version 1.5.18.0 onwards, there’s a new top level namespace Amazon.DynamoDBv2 which contains a mirror set of types to those under the original Amazon.DynamoDB namespace, albeit with minor changes to support the new LSI feature.
Due to the changes in the underlying AWSSDK, I have decided to make some changes to the query syntax supported by DynamoDB.SQL too – namely, to remove the need for the special keywords @HashKey and @RangeKey and instead allow you to use the attributes names for your hash and range keys.
For example, given a table like the one outlined in the DynamoDB docs:
To write a query to find all subjects starting with “a” in the “S3” forum, you would previously write:
SELECT * FROM Thread WHERE @HashKey = \”S3\” AND @RangeKey BEGINS WITH \”a\”
In the new version of DynamoDB.SQL, you would write the following instead:
SELECT * FROM Thread WHERE ForumName = \”S3\” AND Subject BEGINS WITH \”a\”
This syntax change only applies to the extension methods for the AmazonDynamoDBClient and DynamoDBContext types under the new Amazon.DynamoDBv2 namespace. The extension methods themselves are only available under a new namespace DynamoDbV2.SQL.Execution in the DynamoDb.SQL.dll.
The syntax for scans on the other hand, has remained the same in both the new and the old API.
You can specify that a query should use a Local Secondary Index (LSI) by using the Index option in the WITH clause.
For example, given a Thread table and an index LastPostIndex, as outlined in the DynamoDB docs:
To find all the posts in the “S3” forum since the 1st May 2013, you can write the query as following:
SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013–05-01\”
WITH (Index(LastPostIndex, true))
The WITH clause is where you specify optional query parameters, such as NoConsistentRead, and PageSize. (please refer to the Getting Started guide on available query parameters).
The Index option allows you to specify the name of the index, in this case that’s “LastPostIndex”, and a boolean flag to specify whether or not all attributes should be returned.
For the above query, because we’re asking for all attributes to be sent back with *, and that attributes such as Replies are not projected into the index, they will be fetched (automatically performed by DynamoDB) from the main table at additional consumed capacity units.
On the other hand, if you want only the projected attributes back from the index, we can tweak the query slightly:
SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013–05-01\”
WITH (Index(LastPostIndex, false))
In which case, only ForumName, LastPostDateTime and Subject will be returned by the query.
Finally, if you are interested in a specific set of attributes, you can also specify them in the SELECT clause:
SELECT ForumName, Subject FROM Thread
WHERE ForumName = \”S3\” AND LastPostDateTime >= \“2013–05-01\”
WITH (Index(LastPostIndex, false))
Some reference links:
AWS announces Local Secondary Index support for DynamoDB
DynamoDB docs on Local Secondary Indexes
Querying with an Index attribute in DynamoDB.SQL
Getting started with DynamoDB.SQL
Hi, just a quick note to mention that I’ll be talking to Community for F# on the 14th May at 11AM PDT about our experience of building an MMORPG, some of the challenges we face along the way and how F# helps us solve some of these problems.
You can register for the event here.
Hope to see you there!
p.s. here is some of the things that have been happening in Here Be Monsters so far, feel free to come join us in the game 
.Net 4 introduced the Lazy<T> type which allows you to create an object that can be lazily initialized so that you can delay the creation of large objects, for instance.
However, if your initialization logic has the potential to except at runtime (e.g. time out exceptions reading from some external data source) then you should pay close attention to which constructor you use to create a new instance of the Lazy<T> type. Depending on the selected LazyThreadSafetyMode, exceptions in the initialization code might be cached and rethrown on all subsequent attempts to fetch the lazily initialized value. Whilst this ensures that threads will always get the same result, hence removing ambiguity, it does mean that you’ve got only one shot at initializing that value…
In cases where you need to be able to tolerate occasional initialization errors (e.g. reading a large object from S3 can fail from time to time for a number of reasons) and be able to try again at a second attempt, the rule of thumb is to instantiate the Lazy<T> type by setting LazyThreadSafetyMode to PublicationOnly. In PublicationOnly thread safety mode, multiple threads can invoke the initialization logic but the first thread to complete the initialization successfully sets the value of the Lazy<T> instance.
For example, the following only works under the PublicationOnly mode:
F# provides a slightly nicer syntax for defining a lazy computation:
the Control.Lazy<T> type is an abbreviation of the BCL Lazy<T> type with a Force extension method which under the hood just calls Lazy<T>.Value.
Presumably the above translates roughly to the following C# code:
var x = 10;
var result = new Lazy<int>(() => x + 10);
and the thread safety mode using the Lazy(Func<T>) constructor is LazyThreadSafetyMode.ExecutionAndPublication which caches and rethrows any exceptions caught in the initialization. E.g.:
After a long Easter holiday filled with late night coding sessions I find myself wide awake at 2am… good job I’ve still got my pluralsight subscription and a quick look at the Algorithms and Data Structures course again at least gave me something to do to relax the mind with some back-to-school style implementation of common sorting algorithms in F#:
Whilst not the most performant implementations I’m sure, I hope at least it goes to show how easily and concisely these simple algorithms can be expressed in F#! Now back to that sleep thing…
