Echo websocket server using websocketd – as if by magic!

I stumbled across this little gem the other day – websocketd – which turns anything that takes standard-in and standard-out into a websocket server!

To build a dead simple echo server, follow these steps:

  1. follow the download instructions here (don’t forget to add it to your PATH)
  2. create a new console application in Visual Studio (let’s call the console app EchoServer), something like this will suffice:

  1. run this in command line:

and voila!

 

Since we enabled the dev console with the devconsole flag we can now go to http://localhost:8080 in the browser and test out our echo websocket server interactively:

image

pretty sweet, right?

 

One thing to keep in mind though before you consider websocketd for any serious development work, is its implementation details. Based on this paragraph from its project page:

Upon startup, websocketd will start a WebSocket server on a specified port, and listen for connections.

Upon a connection, it will fork the appropriate process, and disconnect the process when the WebSocket connection closes (and vice-versa).

Any message sent from the WebSocket client will be piped to the process’s STDIN stream, followed by a \n newline.

Any text printed by the process to STDOUT shall be sent as a WebSocket message whenever a \n newline is encountered.

in our case this means every time a new connection is establish websocketd will start a new instance of our console app which in practice is unlikely to be what we actually want in a websocket application such as a multiplayer game, and in any production system forking will become really expensive really quickly!

Keep in mind though that it’s still a work-in-progress so the implementation details might change in the future.

 

Links

DynamoDB.SQL 1.2.1 – now supports Local Secondary Index

A couple of weeks earlier, Amazon announced support for Local Secondary Indexes (LSI) for DynamoDB. You can now perform fast, efficient queries against DynamoDB tables using attributes that are not part of the existing Hash and Range key model without resorting to the use of scans.

As a result to the new feature the DynamoDB query API has also gone through some changes, as did the AWSSDK for .Net. From version 1.5.18.0 onwards, there’s a new top level namespace Amazon.DynamoDBv2 which contains a mirror set of types to those under the original Amazon.DynamoDB namespace, albeit with minor changes to support the new LSI feature.

Query syntax change

Due to the changes in the underlying AWSSDK, I have decided to make some changes to the query syntax supported by DynamoDB.SQL too – namely, to remove the need for the special keywords @HashKey and @RangeKey and instead allow you to use the attributes names for your hash and range keys.

For example, given a table like the one outlined in the DynamoDB docs:

image

To write a query to find all subjects starting with “a” in the “S3” forum, you would previously write:

SELECT * FROM Thread WHERE @HashKey = \”S3\” AND @RangeKey BEGINS WITH \”a\”

In the new version of DynamoDB.SQL, you would write the following instead:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND Subject BEGINS WITH \”a\”

This syntax change only applies to the extension methods for the AmazonDynamoDBClient and DynamoDBContext types under the new Amazon.DynamoDBv2 namespace. The extension methods themselves are only available under a new namespace DynamoDbV2.SQL.Execution in the DynamoDb.SQL.dll.

The syntax for scans on the other hand, has remained the same in both the new and the old API.

Local Secondary Index support

You can specify that a query should use a Local Secondary Index (LSI) by using the Index option in the WITH clause.

For example, given a Thread table and an index LastPostIndex, as outlined in the DynamoDB docs:

image

To find all the posts in the “S3” forum since the 1st May 2013, you can write the query as following:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013-05-01\”

WITH (Index(LastPostIndex, true))

The WITH clause is where you specify optional query parameters, such as NoConsistentRead, and PageSize. (please refer to the Getting Started guide on available query parameters).

The Index option allows you to specify the name of the index, in this case that’s “LastPostIndex”, and a boolean flag to specify whether or not all attributes should be returned.

For the above query, because we’re asking for all attributes to be sent back with *, and that attributes such as Replies are not projected into the index, they will be fetched (automatically performed by DynamoDB) from the main table at additional consumed capacity units.

 

On the other hand, if you want only the projected attributes back from the index, we can tweak the query slightly:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013-05-01\”

WITH (Index(LastPostIndex, false))

In which case, only ForumName, LastPostDateTime and Subject will be returned by the query.

 

Finally, if you are interested in a specific set of attributes, you can also specify them in the SELECT clause:

SELECT ForumName, Subject FROM Thread

WHERE ForumName = \”S3\” AND LastPostDateTime >= \“2013-05-01\”

WITH     (Index(LastPostIndex, false))

 

Some reference links:

AWS announces Local Secondary Index support for DynamoDB

DynamoDB docs on Local Secondary Indexes

DynamoDB docs on Query

Querying with an Index attribute in DynamoDB.SQL

Getting started with DynamoDB.SQL

Be Lazy, but be ware of initialization exception

.Net 4 introduced the Lazy<T> type which allows you to create an object that can be lazily initialized so that you can delay the creation of large objects, for instance.

However, if your initialization logic has the potential to except at runtime (e.g. time out exceptions reading from some external data source) then you should pay close attention to which constructor you use to create a new instance of the Lazy<T> type. Depending on the selected LazyThreadSafetyMode, exceptions in the initialization code might be cached and rethrown on all subsequent attempts to fetch the lazily initialized value. Whilst this ensures that threads will always get the same result, hence removing ambiguity, it does mean that you’ve got only one shot at initializing that value…

 

LazyThreadSafetyMode

In cases where you need to be able to tolerate occasional initialization errors (e.g. reading a large object from S3 can fail from time to time for a number of reasons) and be able to try again at a second attempt, the rule of thumb is to instantiate the Lazy<T> type by setting LazyThreadSafetyMode to PublicationOnly. In PublicationOnly thread safety mode, multiple threads can invoke the initialization logic but the first thread to complete the initialization successfully sets the value of the Lazy<T> instance.

For example, the following only works under the PublicationOnly mode:

 

F#

F# provides a slightly nicer syntax for defining a lazy computation:

image

the Control.Lazy<T> type is an abbreviation of the BCL Lazy<T> type with a Force extension method which under the hood just calls Lazy<T>.Value.

Presumably the above translates roughly to the following C# code:

var x = 10;

var result = new Lazy<int>(() => x + 10);

and the thread safety mode using the Lazy(Func<T>) constructor is LazyThreadSafetyMode.ExecutionAndPublication which caches and rethrows any exceptions caught in the initialization. E.g.:

image

Sorting Algorithms in F#

After a long Easter holiday filled with late night coding sessions I find myself wide awake at 2am… good job I’ve still got my pluralsight subscription and a quick look at the Algorithms and Data Structures course again at least gave me something to do to relax the mind with some back-to-school style implementation of common sorting algorithms in F#:

Whilst not the most performant implementations I’m sure, I hope at least it goes to show how easily and concisely these simple algorithms can be expressed in F#! Now back to that sleep thing…

F# – XmlSerializer, Record types and [CLIMutable]

I talked about the XML and JSON serialization of F# types in a previous post, and since then F# 3.0 has been released and a new [CLIMutable] was quietly added amidst the hype surrounding the awesome type providers!

 

CLIMutable Attribute

When applied to a F# record type, the CLIMutable attribute tells the F# compiler to generate a default parameterless constructor as well as property getters and setters so that when working with the record type from other .Net languages it’ll appear as a pretty standard, mutable CLI type that you’d expect.

However, when working in F#, the normal rules as far as record types are concerned still apply – immutability by default, pattern matching, cloning using the with keyword, etc.

For a simple record type Person, let’s have a look at the compile code with and without the CLIMutable attribute:

image image

Without the CLIMutable attribute, this is how the record type is normally compiled:

image

With the CLIMutable attribute applied, things look a lot different:

image

That’s all well and good, but why is this useful you might ask?

Because many serializers, such as the BCL’s XmlSerializer (as well as other serializers available in the open source space) only work with types that has a default constructor and contains property setters.

 

XmlSerializer and Record types

Unlike the DataContractSerializer which only requires a type to be decorated with a Serializable or DataContract attribute, the XmlSerializer on the other hand, also requires the type to define a parameterless constructor.

Whilst both serializers generate XML outputs, DataContractSerializer is intended for marshalling data (that conforms to a ‘contract’) for transportation over the wire and therefore less concerned with readability of its output and offers little option for you to customize the generated XML. It’s therefore quite happy to spit out rather unreadable XML for our Person record type above:

 

<FSI_0004.Person xmlns=”http://schemas.datacontract.org/2004/07/”  xmlns:i=”http://www.w3.org/2001/XMLSchema-instance”><Age_x0040_>30</Age_x0040_><Name_x0040_>Yan Cui</Name_x0040_></FSI_0004.Person>

 

Meanwhile, in the land of the XmlSerializer, its sole purpose of existence is to generate XML and offers a platoon of options to help ensure its output meets your functional and aesthetic needs. By default it’ll generate nice, indented output for our CLIMutable version of Person record type:

 

<?xml version=”1.0″?>

<Person xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema”>

  <Name>Yan Cui</Name>

  <Age>30</Age>

</Person>

 

In addition, you can customize the XML output using the normal XML-related attributes:

image

<?xml version=”1.0″?>

<Person xmlns:xsi=”http://www.w3.org/2001/XMLSchema-instance” xmlns:xsd=”http://www.w3.org/2001/XMLSchema” PersonName=”Yan Cui”>

  <PersonAge>30</PersonAge>

</Person>

 

OSS Serializers

Of the OSS JSON serializers I benchmark regularly, JSON.Net (v4.5.11 tested) works with both normal and CLIMutable record types from F#. ServiceStack.Text (v3.9.37 tested) doesn’t work with ‘vanilla’ record types on deserialization (fields are left with default values), but you can workaround by making the fields mutable or add the CLIMutable attribute to the type.

 

I hope this proves to be useful to you, and if like us you have a lot of C# code hanging around just to get pretty XML out of your F# record types, dare I say it’s about to throw away those C# code! Winking smile