Amazon ELB – Some caveats around health check pings

We recently found out about an interesting, undocumented behaviour of Amazon’s Elastic Load Balancing (ELB) service – that health check pings are performed by each and every instance running your ELB service at every health check interval.

Intro to ELB

But first, let me fill in some background information for readers who are not familiar with the various services that are part of the AWS ecosystem.

ELB is a managed load balancing service which automatically scales up and down based on traffic. You can really easily setup periodic health check pings against your EC2 instances to ensure that requests are only routed to availability zones and instances that are deemed healthy based on the results of the pings.

In addition, you can use ELB’s health checks in conjunction with Amazon’s Auto Scaling service to ensure that instances that repeatedly fails healthy checks with a fresh, new instance.

Generally speaking, the ELB health check should ping an endpoint on your web service (and not the default IIS page…) so that a successful ping can at least inform you that your service is still running. However, given the number of external services our game servers typically depend on (distributed cache cluster, other AWS services, Facebook, etc.) we made the decision to make our ping handlers do more extensive checks to make sure that it can still communicate with those services (which might not be the case if the instance is experiencing networking/hardware failures).

Some ELB caveats

However, we noticed that at peak times, our cluster of Couchbase nodes are hit with a thousand pings at exactly the same time from our 100+ game servers at every ELB health check interval and the number of hits just didn’t make sense to us! Working with the AWS support team, who were very helpful and shared the ELB logs with us which revealed that 8 healthy check pings were issued against each game server at each interval.

It turns out that, each instance that runs your ELB service (the number of instances varies depending on the current load) will ping each of your instances behind the load balancer once per health check interval at exactly the same time.

A further caveat being that for an instance to be considered unhealthy it needs to fail the required number of consecutive pings from an individual ELB instance’s perspective. Which means, it’s possible for your instance to be considered unhealthy from the ELB’s point of view without it having failed the required number of consecutive pings from the instance’s perspective.

To help us visualize this problem, suppose there are currently 4 instances running the ELB service for your environment, labelled ELB inst 1-4 below. Let’s assume that the instance behind the ELB always receive pings from the ELB instances in the same order, from ELB inst 1 to 4. So from our instance’s point of view, it receives pings from ELB inst 1, then ELB inst 2, then ELB inst 3 and so on.

Each of these instances will ping your instances once per health check interval. In the last 2 intervals, the instance failed to respond in a timely fashion to the ping by ELB inst 3, but responded successfully to all other pings. So from our instance’s point of view it has failed 2 out of 8 pings, but not consecutively, however, from ELB inst 3’s perspective the instance has failed 2 consecutive pings and should therefore be considered as unhealthy and stop receiving requests until it passes the required number of consecutive pings.

image

Since the implementation details of the ELB is abstracted away from us (and rightly so!) it’s difficult for us to test what happens when this happens – whether or not all other ELB instances will straight away stop routing traffic to that instance; or if it’ll only impact the routing choices made by ELB inst 3.

From an implementation point of view, I can understand why it was implemented this way, with simplicity being the likely answer and it does cover all but the most unlikely of events. However, from the end-user’s point of view of a service that’s essentially a black-box, the promised (or at the very least the expected) behaviour of the service is different from what happens in reality, albeit subtly, and that some implicit assumptions were made about what we will be doing in the ping handler.

The behaviours we expected from the ELB were:

  • ELB will ping our servers at every health check interval
  • ELB will mark instance as unhealthy if it fails x number of consecutive health checks

What actually happens is:

  • ELB will ping our servers numerous times at every health check interval depending on the number of ELB instances
  • ELB will mark instance as unhealthy if it fails x number of consecutive health checks by a particular ELB instance

Workarounds

If there are expensive health checks that you would like to perform on your service but you still like to use the ELB health check mechanism to stop traffic from being routed to bad instances and have the Auto Scaling service replace them instead. One simple workaround would be to perform the expensive operations in a timer event which you can control, and let your ping handler simply respond with HTTP 200 or 500 status code depending on the result of the last internal health check.

S3 – Masterclass Webinar slides

I stumbled across a set of slides with a rather comprehensive overview of the different aspects of S3, worthwhile reading for anyone who works with Amazon S3 regularly. Enjoy!

DynamoDB.SQL 1.2.1 – now supports Local Secondary Index

A couple of weeks earlier, Amazon announced support for Local Secondary Indexes (LSI) for DynamoDB. You can now perform fast, efficient queries against DynamoDB tables using attributes that are not part of the existing Hash and Range key model without resorting to the use of scans.

As a result to the new feature the DynamoDB query API has also gone through some changes, as did the AWSSDK for .Net. From version 1.5.18.0 onwards, there’s a new top level namespace Amazon.DynamoDBv2 which contains a mirror set of types to those under the original Amazon.DynamoDB namespace, albeit with minor changes to support the new LSI feature.

Query syntax change

Due to the changes in the underlying AWSSDK, I have decided to make some changes to the query syntax supported by DynamoDB.SQL too – namely, to remove the need for the special keywords @HashKey and @RangeKey and instead allow you to use the attributes names for your hash and range keys.

For example, given a table like the one outlined in the DynamoDB docs:

image

To write a query to find all subjects starting with “a” in the “S3” forum, you would previously write:

SELECT * FROM Thread WHERE @HashKey = \”S3\” AND @RangeKey BEGINS WITH \”a\”

In the new version of DynamoDB.SQL, you would write the following instead:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND Subject BEGINS WITH \”a\”

This syntax change only applies to the extension methods for the AmazonDynamoDBClient and DynamoDBContext types under the new Amazon.DynamoDBv2 namespace. The extension methods themselves are only available under a new namespace DynamoDbV2.SQL.Execution in the DynamoDb.SQL.dll.

The syntax for scans on the other hand, has remained the same in both the new and the old API.

Local Secondary Index support

You can specify that a query should use a Local Secondary Index (LSI) by using the Index option in the WITH clause.

For example, given a Thread table and an index LastPostIndex, as outlined in the DynamoDB docs:

image

To find all the posts in the “S3” forum since the 1st May 2013, you can write the query as following:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013-05-01\”

WITH (Index(LastPostIndex, true))

The WITH clause is where you specify optional query parameters, such as NoConsistentRead, and PageSize. (please refer to the Getting Started guide on available query parameters).

The Index option allows you to specify the name of the index, in this case that’s “LastPostIndex”, and a boolean flag to specify whether or not all attributes should be returned.

For the above query, because we’re asking for all attributes to be sent back with *, and that attributes such as Replies are not projected into the index, they will be fetched (automatically performed by DynamoDB) from the main table at additional consumed capacity units.

 

On the other hand, if you want only the projected attributes back from the index, we can tweak the query slightly:

SELECT * FROM Thread WHERE ForumName = \”S3\” AND LastPostDateTime >= \”2013-05-01\”

WITH (Index(LastPostIndex, false))

In which case, only ForumName, LastPostDateTime and Subject will be returned by the query.

 

Finally, if you are interested in a specific set of attributes, you can also specify them in the SELECT clause:

SELECT ForumName, Subject FROM Thread

WHERE ForumName = \”S3\” AND LastPostDateTime >= \“2013-05-01\”

WITH     (Index(LastPostIndex, false))

 

Some reference links:

AWS announces Local Secondary Index support for DynamoDB

DynamoDB docs on Local Secondary Indexes

DynamoDB docs on Query

Querying with an Index attribute in DynamoDB.SQL

Getting started with DynamoDB.SQL

DynamoDB.SQL – version 1.1.0 released

Just a quick note to say that another minor update to DynamoDB.SQL has been release, you can view the release notes here.

 

The latest update adds support for a TSQL style WITH keyword for specifying optional parameters for tweaking the query/scan operation. For queries, you can specify the NoConsistentRead and PageSize options to use eventually consistent read and throttle the number of items returned per request respectively. Similarly for scans, you can use the PageSize option for throttling your scan requests too, but the DynamoDB scans does not support strong consistency.

 

According to DynamoDB best practices, you should avoid sudden bursts of read activity, using the new PageSize option you can make sure that your query/scan does not consume too many read capacity unit in a short burst and end up causing more critical reads to be throttled.

 

For example, a query which returns 10 items per request using eventually consistent read will look something like this:

image

whereas a scan will look like:

image

For more details about the full syntax, please refer to the Getting Started document, which has been updated to include the new WITH keyword.

 

Enjoy!

Introduction to AWS SimpleWorkflow Extensions Part 3 – Parallelizing activities

The series so far:

  1. Hello World example
  2. Beyond Hello World

 

Within a workflow, not all activities have to be performed sequentially. In fact, to increase throughput and/or reduce the overall time required to finish a workflow, you might want to perform several activities in parallel provided that they don’t have any inter-dependencies and can be performed independently.

In this post we’re going to see how we can use the SWF extensions library to parallelize activities by scheduling several activity tasks at a single step and then aggregate their results into a singular input to the next activity in the workflow.

image

The parallelized activities receive their input from either:

  1. the workflow execution’s input if this is the first step of the workflow, or
  2. the result of the preceding activity/child workflow in the workflow

As of now, the library requires you to specify a ‘reducer’ which is responsible for aggregating the results of the parallel activities into a single string which is returned as the result of the step in the workflow. There are some caveats to this reducer function (of signature Dictionary<int, string> –> string right now) as you will see in the example, I’ll look to address these oddities and clean up the API in future versions of the library, so please bear with me for now.

The aggregate result of these parallel activities can then be passed along as the input to the subsequent activity as per the example in my previous post.

Example : Count HTML element types

Suppose that, given a URL, you want to count the number of different HTML elements (e.g. <div>, <span>, …) the returned HTML contains, the counting of each element type is independent and can be carried out in parallel. For nicety, we can add an echo activity before and after the count activities so that we can print the input URL and the results to the screen. So you will end up with a workflow that perhaps looks like this:

image

The implementation of this workflow is as follows:

Untitled

 1: #r "bin/Release/AWSSDK.dll"
 2: #r "bin/Release/SWF.Extensions.Core.dll"
 3: 
 4: open Amazon.SimpleWorkflow
 5: open Amazon.SimpleWorkflow.Extensions
 6: 
 7: open System.Collections.Generic
 8: open System.Net
 9: 
10: let echo str = printfn "%s" str; str
11: 
12: // a function to count the number of occurances of a pattern inside the HTML returned
13: // by the specified URL address
14: let countMatches (pattern : string) (address : string) =
15:     let webClient = new WebClient()
16:     let html = webClient.DownloadString address
17: 
18:     seq { 0..html.Length - pattern.Length }
19:     |> Seq.map (fun i -> html.Substring(i, pattern.Length))
20:     |> Seq.filter ((=) pattern)
21:     |> Seq.length
22: 
23: let echoActivity = Activity(
24:                         "echo", "echo input", echo,
25:                         taskHeartbeatTimeout       = 60, 
26:                         taskScheduleToStartTimeout = 10,
27:                         taskStartToCloseTimeout    = 10, 
28:                         taskScheduleToCloseTimeout = 20)
29: 
30: let countDivs = Activity<string, int>(
31:                         "count_divs", "count the number of <div> elements", 
32:                         countMatches "<div",
33:                         taskHeartbeatTimeout       = 60, 
34:                         taskScheduleToStartTimeout = 10,
35:                         taskStartToCloseTimeout    = 10, 
36:                         taskScheduleToCloseTimeout = 20)
37: 
38: let countScripts = Activity<string, int>(
39:                         "count_scripts", "count the number of <script> elements", 
40:                         countMatches "<script",
41:                         taskHeartbeatTimeout       = 60, 
42:                         taskScheduleToStartTimeout = 10,
43:                         taskStartToCloseTimeout    = 10, 
44:                         taskScheduleToCloseTimeout = 20)
45: 
46: let countSpans = Activity<string, int>(
47:                         "count_spans", "count the number of <span> elements", 
48:                         countMatches "<span",
49:                         taskHeartbeatTimeout       = 60, 
50:                         taskScheduleToStartTimeout = 10,
51:                         taskStartToCloseTimeout    = 10, 
52:                         taskScheduleToCloseTimeout = 20)
53: 
54: let countActivities = [| countDivs      :> ISchedulable
55:                          countScripts   :> ISchedulable
56:                          countSpans     :> ISchedulable |]
57: 
58: let countReducer (results : Dictionary<int, string>) =
59:     sprintf "Divs : %d\nScripts : %d\nSpans : %d\n" (int results.[0]) (int results.[1]) (int results.[2])
60: 
61: let countElementsWorkflow = 
62:     Workflow(domain = "theburningmonk.com", name = "count_html_elements", 
63:              description = "this workflow counts", 
64:              version = "1")
65:     ++> echoActivity
66:     ++> (countActivities, countReducer)
67:     ++> echoActivity
68: 
69: let awsKey      = "PUT-YOUR-AWS-KEY-HERE"
70: let awsSecret   = "PUT-YOUR-AWS-SECRET-HERE"
71: let client = new AmazonSimpleWorkflowClient(awsKey, awsSecret)
72: 
73: countElementsWorkflow.Start(client)

namespace Amazon
namespace Amazon.SimpleWorkflow
namespace Amazon.SimpleWorkflow.Extensions
namespace System
namespace System.Collections
namespace System.Collections.Generic
namespace System.Net
val echo : str:string -> string

Full name: ParallelizeActivities.echo

val str : string
val printfn : format:Printf.TextWriterFormat<‘T> -> ‘TFull name: Microsoft.FSharp.Core.ExtraTopLevelOperators.printfn
val countMatches : pattern:string -> address:string -> intFull name: ParallelizeActivities.countMatches
val pattern : string
Multiple items

val string : value:’T -> stringFull name: Microsoft.FSharp.Core.Operators.string

——————–

type string = System.String

Full name: Microsoft.FSharp.Core.string

val address : string
val webClient : WebClient
Multiple items

type WebClient =  inherit Component

new : unit -> WebClient

member AllowReadStreamBuffering : bool with get, set

member AllowWriteStreamBuffering : bool with get, set

member BaseAddress : string with get, set

member CachePolicy : RequestCachePolicy with get, set

member CancelAsync : unit -> unit

member Credentials : ICredentials with get, set

member DownloadData : address:string -> byte[] + 1 overload

member DownloadDataAsync : address:Uri -> unit + 1 overload

member DownloadDataTaskAsync : address:string -> Task<byte[]> + 1 overload

Full name: System.Net.WebClient

——————–

WebClient() : unit

val html : string
WebClient.DownloadString(address: System.Uri) : string

WebClient.DownloadString(address: string) : string

Multiple items

val seq : sequence:seq<‘T> -> seq<‘T>Full name: Microsoft.FSharp.Core.Operators.seq

——————–

type seq<‘T> = IEnumerable<‘T>

Full name: Microsoft.FSharp.Collections.seq<_>

property System.String.Length: int
module Seqfrom Microsoft.FSharp.Collections
val map : mapping:(‘T -> ‘U) -> source:seq<‘T> -> seq<‘U>Full name: Microsoft.FSharp.Collections.Seq.map
val i : int
System.String.Substring(startIndex: int) : string

System.String.Substring(startIndex: int, length: int) : string

val filter : predicate:(‘T -> bool) -> source:seq<‘T> -> seq<‘T>Full name: Microsoft.FSharp.Collections.Seq.filter
val length : source:seq<‘T> -> intFull name: Microsoft.FSharp.Collections.Seq.length
val echoActivity : Activity<string,string>Full name: ParallelizeActivities.echoActivity
Multiple items

type Activity = Activity<string,string>Full name: Amazon.SimpleWorkflow.Extensions.Activity

——————–

new : name:obj * description:obj * processor:System.Func<‘TInput,’TOutput> * taskHeartbeatTimeout:obj * taskScheduleToStartTimeout:obj * taskStartToCloseTimeout:obj * taskScheduleToCloseTimeout:obj * ?taskList:obj -> Activity<‘TInput,’TOutput>

new : name:string * description:string * processor:(‘TInput -> ‘TOutput) * taskHeartbeatTimeout:Model.Seconds * taskScheduleToStartTimeout:Model.Seconds * taskStartToCloseTimeout:Model.Seconds * taskScheduleToCloseTimeout:Model.Seconds * ?taskList:string * ?maxAttempts:int -> Activity<‘TInput,’TOutput>

val countDivs : Activity<string,int>Full name: ParallelizeActivities.countDivs
Multiple items

val int : value:’T -> int (requires member op_Explicit)Full name: Microsoft.FSharp.Core.Operators.int

——————–

type int = int32

Full name: Microsoft.FSharp.Core.int

——————–

type int<‘Measure> = int

Full name: Microsoft.FSharp.Core.int<_>

val countScripts : Activity<string,int>Full name: ParallelizeActivities.countScripts
val countSpans : Activity<string,int>Full name: ParallelizeActivities.countSpans
val countActivities : ISchedulable []Full name: ParallelizeActivities.countActivities
type ISchedulable =

interface    abstract member Description : string

abstract member MaxAttempts : int

abstract member Name : string

end

Full name: Amazon.SimpleWorkflow.Extensions.ISchedulable

val countReducer : results:Dictionary<int,string> -> stringFull name: ParallelizeActivities.countReducer
val results : Dictionary<int,string>
Multiple items

type Dictionary<‘TKey,’TValue> =  new : unit -> Dictionary<‘TKey, ‘TValue> + 5 overloads

member Add : key:’TKey * value:’TValue -> unit

member Clear : unit -> unit

member Comparer : IEqualityComparer<‘TKey>

member ContainsKey : key:’TKey -> bool

member ContainsValue : value:’TValue -> bool

member Count : int

member GetEnumerator : unit -> Enumerator<‘TKey, ‘TValue>

member GetObjectData : info:SerializationInfo * context:StreamingContext -> unit

member Item : ‘TKey -> ‘TValue with get, set

nested type Enumerator

nested type KeyCollection

nested type ValueCollection

Full name: System.Collections.Generic.Dictionary<_,_>

——————–

Dictionary() : unit

Dictionary(capacity: int) : unit

Dictionary(comparer: IEqualityComparer<‘TKey>) : unit

Dictionary(dictionary: IDictionary<‘TKey,’TValue>) : unit

Dictionary(capacity: int, comparer: IEqualityComparer<‘TKey>) : unit

Dictionary(dictionary: IDictionary<‘TKey,’TValue>, comparer: IEqualityComparer<‘TKey>) : unit

val sprintf : format:Printf.StringFormat<‘T> -> ‘TFull name: Microsoft.FSharp.Core.ExtraTopLevelOperators.sprintf
val countElementsWorkflow : WorkflowFull name: ParallelizeActivities.countElementsWorkflow
Multiple items

type Workflow =  interface IWorkflow

new : domain:string * name:string * description:string * version:string * ?taskList:string * ?stages:Stage list * ?taskStartToCloseTimeout:Seconds * ?execStartToCloseTimeout:Seconds * ?childPolicy:ChildPolicy * ?identity:Identity * ?maxAttempts:int -> Workflow

member private Append : toStageAction:(‘a -> StageAction) * args:’a -> Workflow

member Start : swfClt:AmazonSimpleWorkflowClient -> unit

member add_OnActivityFailed : Handler<Domain * Name * ActivityId * Details option * Reason option> -> unit

member add_OnActivityTaskError : Handler<Exception> -> unit

member add_OnDecisionTaskError : Handler<Exception> -> unit

member add_OnWorkflowCompleted : Handler<Domain * Name> -> unit

member add_OnWorkflowFailed : Handler<Domain * Name * RunId * Details option * Reason option> -> unit

member NumberOfStages : int

Full name: Amazon.SimpleWorkflow.Extensions.Workflow

——————–

new : domain:string * name:string * description:string * version:string * ?taskList:string * ?stages:Stage list * ?taskStartToCloseTimeout:Model.Seconds * ?execStartToCloseTimeout:Model.Seconds * ?childPolicy:Model.ChildPolicy * ?identity:Model.Identity * ?maxAttempts:int -> Workflow

val awsKey : stringFull name: ParallelizeActivities.awsKey
val awsSecret : stringFull name: ParallelizeActivities.awsSecret
val client : AmazonSimpleWorkflowClientFull name: ParallelizeActivities.client
Multiple items

type AmazonSimpleWorkflowClient =  inherit AmazonWebServiceClient

new : unit -> AmazonSimpleWorkflowClient + 11 overloads

member BeginCountClosedWorkflowExecutions : countClosedWorkflowExecutionsRequest:CountClosedWorkflowExecutionsRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginCountOpenWorkflowExecutions : countOpenWorkflowExecutionsRequest:CountOpenWorkflowExecutionsRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginCountPendingActivityTasks : countPendingActivityTasksRequest:CountPendingActivityTasksRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginCountPendingDecisionTasks : countPendingDecisionTasksRequest:CountPendingDecisionTasksRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginDeprecateActivityType : deprecateActivityTypeRequest:DeprecateActivityTypeRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginDeprecateDomain : deprecateDomainRequest:DeprecateDomainRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginDeprecateWorkflowType : deprecateWorkflowTypeRequest:DeprecateWorkflowTypeRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginDescribeActivityType : describeActivityTypeRequest:DescribeActivityTypeRequest * callback:AsyncCallback * state:obj -> IAsyncResult

member BeginDescribeDomain : describeDomainRequest:DescribeDomainRequest * callback:AsyncCallback * state:obj -> IAsyncResult

Full name: Amazon.SimpleWorkflow.AmazonSimpleWorkflowClient

——————–

AmazonSimpleWorkflowClient() : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(region: Amazon.RegionEndpoint) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(config: AmazonSimpleWorkflowConfig) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(credentials: Amazon.Runtime.AWSCredentials) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(credentials: Amazon.Runtime.AWSCredentials, region: Amazon.RegionEndpoint) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(credentials: Amazon.Runtime.AWSCredentials, clientConfig: AmazonSimpleWorkflowConfig) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(awsAccessKeyId: string, awsSecretAccessKey: string) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(awsAccessKeyId: string, awsSecretAccessKey: string, region: Amazon.RegionEndpoint) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(awsAccessKeyId: string, awsSecretAccessKey: string, clientConfig: AmazonSimpleWorkflowConfig) : unit

(+0 other overloads)

AmazonSimpleWorkflowClient(awsAccessKeyId: string, awsSecretAccessKey: string, awsSessionToken: string) : unit

(+0 other overloads)

member Workflow.Start : swfClt:AmazonSimpleWorkflowClient -> unit

Thanks to Tomas Petricek’s FSharp.Formatting project I’m now able to provide code snippets with intellisense! Tomas, you rock!

 

Running the above example and starting a workflow execution with the input

http://www.bing.com

outputs the following to the console:

image

If you take a look at the history of events below, the decision task following the completion of stage 0 (the first echo activity) was completed with the state (tracked in the Execution Context property):

{“CurrentStageNumber”:1,”NumberOfActions”:3,”Results”:{}}

Without going into too much details on the inner workings of the generated decider, this JSON serialized state tells us that the workflow has moved into stage no. 1, where there are a total of 3 actions, each represented by an activity task to count a particular type of HTML element.

image

Switching to the Activities tab, you can see that 3 activities were completed at stage index 1 of the count_html_elements workflow, judging by the Activity ID and Version of the 3 activities:

image

Caveats

Looking at the example code, a couple of questions jump out straight away:

Q. What is the ISchedulable interface?

The ISchedulable interface represents anything that can be scheduled as a part of a workflow, i.e. an activity or a child workflow. Both IActivity and IWorkflow inherits from it though in all the examples so far we’ve only worked directly against the concept implementation types for these two interfaces.

 

Q. Why does the reducer take a Dictionary<int, string>?

A. As far as the reducer is concerned, it probably doesn’t need to be. The main reason I’ve used a dictionary here is that when intermediate results are available (e.g. 2 out of 3 parallel activities have completed) I wanted to be able to show the current set of results in the Execution Context for the workflow execution (see screenshot above). Because we don’t have all the results back, so I needed to be able to show the result against the originating cativity, hence why a dictionary where the key is the zero-based index of the activity in the input array and the value is the string representation of the result.

 

Q. Why then, are the Dictionary’s value strings when the scheduled activity can be generic and return arbitrary types?

Because not all the activities have to return the same type.

Under the hood the generic Activity<TInput, TOutput> marshals data to and from JSON strings using ServiceStack.Text JSON serializer, and the Activity type is just a special case where both TInput and TOutput are strings.

When the result is recorded and retrieved via SWF, they’re already in string format, although it’s possible to inspect the originating activity’s generic type parameters to work out the returned type, to cater for different return types, the dictionary would need to be a Dictionary<int, object> instead, which is not any better.

 

Q. So what if I want to return anything other than a string from the reducer function?

For now, you can use the ServiceStack.Text JSON serializer (for better compatibility) to serialize the return value to string yourself, I’ll add support for the library to do this automatically in version 1.1.0 release. It skipped my mind at the time, sorry…

 

Next

In the next post, I’ll show you how you can add child workflows in the mix. As I’ve mentioned above, workflows also implement the ISchedulable interface and can be scheduled into the workflow in the same way as activities.

To find out the latest announcements and updates on the Amazon.SimpleWorkflow.Extensions project, please follow the official twitter account @swf_extensions, and as always, your feedbacks and comments on the project will be much appreciated!