A Guide to Building Workflow Based Application with AWS SWF - HashedIn Technologies

A Guide to Building Workflow Based Application with AWS SWF

Technology - 09 Jun 2017
Vinit Kumar

Business workflows are very common in applications and often play the most critical role. In this post, we will explore AWS SWF service to handle business workflows.

Before understanding the SWF, let’s be on the same page by understanding what I mean by a workflow. A Workflow is a sequence of activities which we perform to achieve a goal. Now, sequence of activities may be dynamic and decided based on some inputs, usually the output of previous activity or may be some external signal. We often represent a workflow using flow diagrams. For an example – An online taxi booking system might have a workflow for taxi booking as;

  1. Search for nearby taxis which matches taxi-type criteria of search.
  2. If the search gives the results, then:2.1. Send notification all taxi drivers about the order and wait for the fixed time period for confirmation.

    2.2. If the confirmation came in time, then it sends confirmation to the customer. Else after timeout send SMS to customer to retry after sometime.

  3. Else, it sends a message about non availability of the taxi and asks to retry after some time.

Of course, this is just the booking flow. But, in reality, this will be more complicated for complete order lifecycle. Here is the flowchart for this.

flowchart

You can easily observe the technical challenges involved in handling this simple workflow where we have to maintain the state at each steps to take next decision. Let’s see what AWS SWF provides to solve this.

AWS SWF is a reliable & scalable solution to run jobs that have parallel or sequential steps. It provides task coordination and state tracking and allows you to completely control the decision making and activity functioning. Following are the terminologies for SWF.

  1. Worker: An application which perform some task. There are two type of workers in SWF, Decider & Activity workers. Decider Workers are responsible for performing decisions by taking the state history and returning next activity task to perform or completing the workflow execution. Decider corresponds to diamond box in flow-chart. Activity Workers are responsible for performing the actual task.
  2. Tasks: SWF interacts with workers by providing them some unit of work called task. It can be an activity task which needs to be performed by Activity Worker or decision task which needs to be performed by a decider or a lambda task, which is a special activity task that can be executed using AWS lambda function.
  3. WorkflowType: Every workflow in SWF needs to be registered by providing name and version. This is just to identify a workflow. e.g. for above discussed taxi booking system, we can have a workflow type as ‘TaxiBookingWorkflow’
  4. Domain: Provides a way to scope AWS resource within AWS account. All tasks and workflow needs to be associated with a domain.
  5. Workflow Starter: An application which kicks of the workflow execution. In our taxi booking app, a backend API handler for booking request could be the workflow starter.

Brief on how SWF based application works

  1. Create a domain in SWF and workflow in SWF. And then register activity tasks in the workflow.
  2. Start workflow execution.
  3. Start decider worker which will Poll for decision tasks and find the next step to do.
  4. Start Activity workers which will poll for activity task and perform required task.

Here is a simple diagram explaining how SWF works.

SWFApplicationMode

So, for our above discussed taxi booking system we will,

  1. Register a domain(a name to scope all the related SWF entities), workflow-type(just an identifier to booking workflow) and activities with SWF.
  2. Create decider – python program which keeps polling decider queue and output next step.
  3. Create Activity worker – python program which keeps polling activity queue and executes task.

Let’s see some code in action. For Python, there is no flow framework. So, we will use low level boto3 API’s.

Summary

Of-course the code sample didn’t cover the edge cases and all flows like order cancellation from customer etc. But, hopefully this should have given some idea on for what & how to use AWS SWF. The code might look more verbose because of the uses of low level APIs which I think could be improved with some higher library/framework like flow(currently available only for Java & ruby).

To learn more you can check these online references

Share
Tweet
+1
Share

E-book on Digital Business Transformation