Introduction to AWS SimpleWorkflow Extensions Part 1 – Hello World example

Series so far:

2. Beyond Hel­lo World

3. Par­al­leliz­ing activ­i­ties

 

In my pre­vi­ous post I men­tioned some of the short­com­ings with Ama­zon Sim­ple­Work­flow (SWF) which drove me to cre­ate an exten­sion library on top of the stan­dard .Net SDK to make it eas­i­er to mod­el work­flows and busi­ness process­es using SWF.

In this series of blog posts I’ll give you more exam­ples of how to use the library to mod­el work­flows to be exe­cut­ed against the SWF ser­vice to take advan­tage of the reli­able state man­age­ment and task dis­patch it offers, but none of the plumb­ing and boil­er­plate code you would have to deal with using the SDK.

Before we start look­ing at exam­ples, let’s have a quick recap of the SWF ter­mi­nolo­gies:

  • A work­flow is a sequence of steps that are loose­ly strung togeth­er by the deci­sions the decider makes each time the state of the work­flow changes. E.g. step 1 com­plete then sched­ule step 2 to com­mence.
  • A work­flow exe­cu­tion is an instance of a par­tic­u­lar work­flow cur­rent­ly being exe­cut­ed, many exe­cu­tions of the same work­flow (iden­ti­fied by name and ver­sion) can be in flight at the same time. A work­flow exe­cu­tion can be start­ed with string as input and it can return string as out­put.
  • A deci­sion task is a task that is sched­uled each time a workflow’s state changes.
  • A decider is a com­po­nent in your appli­ca­tion which is respon­si­ble for polling SWF for deci­sion tasks and respond with deci­sions. The sequence of steps that need to be per­formed by the work­flow is ulti­mate­ly deter­mined by the decider.
  • An activ­i­ty task is a task that is sched­uled by a decider, it takes a string as input (along with sev­er­al oth­er pieces of data which it can be sched­uled with) and returns a string as result.
  • An activ­i­ty work­er is a com­po­nent in your appli­ca­tion which is respon­si­ble for polling SWF for activ­i­ty tasks and respond with com­ple­tion or fail­ure sig­nals, as well as pro­vid­ing reg­u­lar heart­beat sig­nals. If the decider is respon­si­ble for sched­ul­ing work to be done, then the activ­i­ty work­er is respon­si­ble for doing the actu­al work.
  • A child work­flow is a work­flow that is sched­uled by the decider as a step in a work­flow, sim­i­lar to an activ­i­ty.
  • The decider is able to sched­ule both child work­flows and activ­i­ties for a sin­gle step in a work­flow, whilst child work­flows can be rerun as an inde­pen­dent unit of work, activ­i­ties can­not be rerun inde­pen­dent­ly out­side of the con­text of a work­flow.
  • Both work­flows and activ­i­ties need to be reg­is­tered with the SWF ser­vice before they can be used.

These are the most com­mon concepts/components you’ll see in SWF, but there are also less com­mon­ly used (in my opin­ion at least) fea­tures such as:

  • Start­ing a timer to cause a timer event to be fired after some time.
  • Sig­nalling an exter­nal work­flow exe­cu­tion to cause an event to be record­ed in its exe­cu­tion his­to­ry and a deci­sion task to be sched­uled. This is a use­ful way to allow inter-work­flow com­mu­ni­ca­tion, e.g. one work­flow sus­pends itself, until anoth­er work­flow sends it a sig­nal and then it can resume with its exe­cu­tion.
  • Record­ing a mark­er as means to pro­vide addi­tion­al infor­ma­tion in the exe­cu­tion his­to­ry of a work­flow.

 

Example : Hello World

Con­sid­er a work­flow where there is only one activ­i­ty, which sim­ply prints the input to the screen and echoes it back out.

If we start a work­flow exe­cu­tion with the input “Hel­lo World!” then we expect to see the input being print­ed to the con­sole and then the work­flow exe­cu­tion com­plet­ed with the result “Hel­lo World!”.

image

In its essence, you can think of an activ­i­ty as noth­ing more than a func­tion which accepts a string as argu­ment and return a string, i.e. a fun with sig­na­ture string –> string in F#.

With the stan­dard .Net SDK you will need to write a decider for each work­flow in order to pro­vide the orches­tra­tion you need for that work­flow. The decider log­ic tends to quick­ly become dif­fi­cult to under­stand and main­tain when the deci­sion log­ic becomes more com­pli­cat­ed, e.g. when mul­ti­ple activ­i­ties and child work­flows are sched­uled in par­al­lel ‚and you need to retry/fail activities/workflows, etc.

In my view, the decider is large­ly plumb­ing that devel­op­ers should do with­out, so with the exten­sions library you should not need to write any cus­tom decider code but instead, sim­ply declare what activ­i­ties and/or child work­flows should be sched­uled at each stage of a work­flow and let the library do all the heavy lift­ing for you!

As far as work­flow mod­el­ling is con­cerned, the only thing you need to do is use the cus­tom ++> oper­a­tor (inspired by Dave Thomas’s pipelets project) to attach addi­tion­al steps to your work­flow. So the above work­flow can be mod­elled as:

and that’s it! No need to reg­is­ter the work­flow and activ­i­ty and write bespoke decider & activ­i­ty work­er your­self, the library does all of that for you, all you need­ed to do was to mod­el the work­flow you want.

Notice you haven’t had to pro­vide any ref­er­ence to SWF at all thus far, in fact, you only need to pro­vide an instance of Ama­zon­Sim­ple­Work­flow­Client (from the AWS SDK) when you start the work­flow:

image

This way, it’s pos­si­ble to run the work­flow across mul­ti­ple accounts simul­ta­ne­ous­ly (dev, stag­ing, prod, etc.) by call­ing the Start method with each of the client instances (one for each account), which fits well with the mobile work­er mod­el SWF is designed with – SWF holds the state but you can run your work­ers from any­where in and out of the AWS ecosys­tem.

Once you’ve start­ed the work­flow, the library will auto­mat­i­cal­ly reg­is­ter the domain, work­flow and activ­i­ty for you if they are not present already. You can ver­i­fy this by look­ing in the SWF Man­age­ment Con­sole:

image

Notice that whilst we didn’t spec­i­fy the “echo” activ­i­ty with a ver­sion num­ber, it’s reg­is­tered with “echo.0”? I’ll go into more details on the ver­sion­ing scheme in a lat­er post, but for now let’s just be glad that we didn’t have to reg­is­ter these by hand!

Next, you can start a work­flow direct­ly from the man­age­ment con­sole, but tick­ing against the work­flow you want to start and click­ing the “Start New Exe­cu­tion” but­ton:

image

Let’s fol­low through with the dia­logue box and set the input as Hel­lo World! as below:

image image

Once you start the work­flow exe­cu­tion you will see Hel­lo World! being print­ed in the con­sole:

image

This is a sign that our echo func­tion (which is invoked by the gen­er­at­ed activ­i­ty work­er) had been called.

Back in the SWF Man­age­ment Con­sole, if you look under Work­flow Exe­cu­tions, you should see the exe­cu­tion is closed after hav­ing com­plet­ed suc­cess­ful­ly:

image

Click­ing on the work­flow exe­cu­tion ID allows you to see the sequence of events which had been record­ed for this exe­cu­tion:

image

This is a very gran­u­lar view of what hap­pened dur­ing the work­flow exe­cu­tion, giv­ing you plen­ty of use­ful infor­ma­tion if you ever need to inves­ti­gate why a work­flow exe­cu­tion failed, for instance.

If you switch to the Activ­i­ties tab, you’ll get a more con­densed view with just the activ­i­ties that were sched­uled, along with their inputs, results, etc.

image

For now, ignore the JSON string in the Con­trol field and the for­mat of the Activ­i­ty ID, these are both auto­mat­i­cal­ly gen­er­at­ed by the library based on a set of con­ven­tions and will be cov­ered by a lat­er post.

So that’s it! I hope you can see that this exten­sion library gives you a pow­er­ful way to express and mod­el a work­flow and focus your devel­op­ment efforts on the things that count (design­ing the process and writ­ing the code that does the actu­al work) rather than wast­ing pre­cious devel­op­er time on get­ting your code to work with SWF!

 

Parting Thoughts

For Java devel­op­ers, there is an exist­ing high-lev­el frame­work (pro­vid­ed by Ama­zon itself) for work­ing with SWF called the Flow Frame­work, which adapts a more object-ori­ent­ed approach and in my opin­ion requires far more plumb­ing and most impor­tant­ly does not

In case you’re won­der­ing, this is how a solu­tion to a sim­i­lar Hel­lo World exam­ple looks using the flow frame­work (tak­en straight from the flow frame­work devel­op­er guide) for your com­par­i­son: