Getting Started with AWS Step Functions

Getting Started with AWS Step Functions

Serverless, deterministic, and traceable Lambda-powered Workflows

Step functions by AWS is a service to seamlessly build an orchestrated flow of Lambda function executions for different business cases. In this article, I want to give you a small introduction to creating delayed notifications for your customers. An example use-case is to send another message if the double opt-in step is not completed after a certain time.

Quick overview of the content:

  • Serverless Step Functions — getting started with Step Functions & Serverless Framework.

  • Triggering our state machine — how to start our state machine & how to integrate Lambda executions by future steps.

  • Adding Metrics & Alerts — adding some visibility to get aware of execution errors.

  • Key Takeaways

As the overview already mentions, all infrastructure will solely be created via Serverless Framework.

Serverless Plugin for Step Functions

The fastest way of getting started is to make use of the Serverless plugin [serverless-step-functions](https://github.com/serverless-operations/serverless-step-functions). It enables you to create state machines with a few lines of code.

Creating the State Machine

The state machine we want to create is pretty simple and only contains two states: wait & trigger notification. The first one will only delay the execution of the next step for a specific number of seconds. The second step will execute another Lambda function — which will in our case send out a second customer notification if there’s still a need for it.

A Simple State Machine in AWS Step Functions

Let’s add this to our serverless template file.

stepFunctions:  
  stateMachines:  
    confirmationReminder:  
      name: confirmation-reminder  
      definition:  
        StartAt: Wait  
        States:  
          Wait:  
            Type: Wait  
            Seconds: 86400 # 1 day  
            Next: TriggerNotification  
          TriggerNotification:  
            Type: Task  
            Resource:  
              Fn::GetAtt: [reminder, Arn]  
            End: true

We did not configure an initial trigger of the state machine, because we want to do this after the registration was submitted by the customer. After submission, the state machine will pause its execution for one day and afterward execute our reminder function.

functions:  
  registration:  
    handler: register.handler  
    environment:  
      STATE_MACHINE_ARN: arn:aws:states:#{AWS::Region}:#{AWS::AccountId}:stateMachine:confirmation-reminder  
    # [...] configuration of everything else :)  
  reminder:  
    handler: reminder.handler  
    # [...]

We’re passing the ARN of the state machine via an environment variable, so we can easily use it in our function (the region & account id is retrieved via serverless-pseudo-parameters) via process.env.STATE_MACHINE_ARN.

Finally, let’s not forget about adding the required IAM action states:StartExecution for our new resource.

provider:  name: aws  
  iam:  
    role:  
      statements:  
        - Effect: "Allow"  
          Action:  
            - states:StartExecution  
          Resource:  
            - arn:aws:states:*:*:stateMachine:confirmation-reminder

And that’s already everything for the infrastructure part.

Triggering our State Machine

Now, as we’ve got our state machine set up successfully, we can start executions on it. I like to create a common interface for the object I’m passing — in this case it includes the customer identifier and the initial trace.

import AWS, { StepFunctions } from 'aws-sdk'
import { v4 } from 'uuid'
import { StartExecutionInput, StartExecutionOutput } from 'aws-sdk/clients/stepfunctions'

interface StateMachineSubmission {
  customerId: string
  traceId: string
}

export class StateMachineService {

  private client: StepFunctions

  constructor() {
    this.client = new AWS.StepFunctions()
  }

  private async startExecution(customerId: string, traceId: string) {
    const payload: StateMachineSubmission = { customerId, traceId }
    const params: StartExecutionInput = {
      stateMachineArn: process.env.STATE_MACHINE_ARN,
      input: JSON.stringify(payload),
      name: `${customerId}_${v4().toString()}`,
    }

    const response = await this.client.startExecution(params).promise()
      .catch((err) => {
        console.err(`State machine could not be started [err=${err}, customerId=${customerId}]`)
      })) as StartExecutionOutput
    if (response) {
      console.log(`StateMachine execution has been started [date=${response.startDate}, arn=${response.executionArn}`)
    }
  }
}

With this, we can start the state machine at each registration. After the wait-task is finished within 24 hours, our reminder function will be executed.

The event we’ll receive in our reminder's handler function will be the payload we passed to the state machine in the beginning — in this case, it contains our customer's identifier. We can check back if the confirmation process of the account was finished — if not, just send out another notification.

💡 There’s an excellent console interface — you can browse and view your state machine executions on the AWS console. There’s even a visual representation of each run, showing you the real-time state of the execution and the state of each task (In Progress, Succeeded, Failed, Cancelled, or Caught Error).

Adding Metrics & Alerts

Surely, we want to be notified if executions of the state machine fail. Serverless also makes this effortless by providing single-line configurations of metrics and corresponding alarms.

stepFunctions:  
  stateMachines:  
    confirmationReminder:  
      name: confirmation-reminder  
      definition:  
        # [...]  
      alarms:  
        topics:  
          ok: arn:aws:sns:us-east-1:1234567890:notify  
          alarm: arn:aws:sns:us-east-1:1234567890:notify  
          insufficientData: arn:aws:sns:us-east-1:1234567890:notify  
        metrics:  
          - executionsTimedOut  
          - executionsFailed  
          - executionsAborted  
          - executionThrottled  
          - executionsSucceeded  
        treatMissingData: missing

If there are any alert state changes now for our state machine, we’ll receive a notification on our notify SNS topic. You’re free to forward those messages via different channels. I like to use AWS Chatbot to forward messages (in a well-formatted output) to Slack. If you’re interested, have a look at one of my previous articles about integration Chatbot via CloudFormation/Terraform.

Key takeaways

Step functions are a neat and simple way to build complex and asynchronous business processes with high reliability due to the use of Lambda functions. By using Serverless Framework, you can easily integrate your first use case with just a few lines of configuration and code.

And as always: there are almost no limits on what you can build with this excellent service. I’m planning to make heavy use of step functions in the future.