There are many use cases where you need to do certain actions based on database changes. For example, each time a new user is added to your database you want to send an automated email to the user. This is a perfect example of the usage of DynamoDB Streams.

DynamoDB Streams can trigger AWS Lambda functions for each database change. For example, if you want to trigger a function once a new user is added to your user table you can enable a DynamoDB Stream. A lambda function will automatically be triggered with the new user.

In this post, we'll show you how to activate streams, which event it will trigger, and show you a common example.

Let's go.

Creating Streams

You can create a stream with any Infrastructure as Code tool (Terraform, CDK, CloudFormation) or with the API or CLI. In this post, we will show you how to create streams with the Management Console.

Enable Streams

You first need to enable streams in the DynamoDB console:

1. Go to your table settings

2. Head over to the tab Exports and streams

3. In the card, DynamoDB stream details click on Enable

4. Define which data you want to consume in your lambda function

There are different options to choose from:

For our example of getting new users, it is enough to choose New Image. With that, we always get the whole user object in the lambda function.

Streams are enabled now ✅

Create a Trigger

Now we need to create a trigger. This connects the DynamoDB Stream to the Lambda function.

1. Go to your table settings

2. Head over to the tab Exports and streams

3. After enabling your streams you should now see a window with **Triggers**

   

4. Click on Create Trigger to create a new trigger

5. 

   In this window, you need to define your lambda function. Either select one you already have or Create a new one. You can also choose a batch size of how many events your lambda should handle. We leave it at one for now.

Make sure your lambda function is allowed to do the following actions:

• `GetRecord`

• `GetShardIterator `

• `DescribeStream `

• `ListStreams `

{
            "Effect": "Allow",
            "Action": [
                "dynamodb:DescribeStream",
                "dynamodb:GetRecords",
                "dynamodb:GetShardIterator",
                "dynamodb:ListStreams"
            ],
            "Resource": "STREAM_ARN"
}

Your Stream ARN looks something like that: arn:aws:dynamodb:eu-central-1:<ACCOUNT_ID>:table/Users/stream/2022-08-21T10:49:51.310

Please add a star behind the stream part so it should look like that: arn:aws:dynamodb:eu-central-1:<ACCOUNT_ID>:table/Users/stream/*

Lambda Function

Your Lambda will now be triggered for every insert, update, and deletion in your DynamoDB Table. Let's head over to the lambda service and work on the function.

So let's go to the lambda function and add the code

exports.handler = async (event) => {
  console.info("EVENT\n" + JSON.stringify(event, null, 2))
    // TODO implement
    const response = {
        statusCode: 200,
        body: JSON.stringify('Hello from Lambda!'),
    };
    return response;
};

After that, we head over to the DynamoDB table and insert an item.

Tip ☝️: You can also head over to this website and check out the event. OR generate a test event within the AWS Lambda console.

{
    "Records": [
        {
            "eventID": "d888c8818651dc51d98b21f921be01fd",
            "eventName": "INSERT",
            "eventVersion": "1.1",
            "eventSource": "aws:dynamodb",
            "awsRegion": "eu-central-1",
            "dynamodb": {
                "ApproximateCreationDateTime": 1661096164,
                "Keys": {
                    "createdAt": {
                        "S": "2022-08-20"
                    },
                    "userId": {
                        "S": "user_2"
                    }
                },
                "NewImage": {
                    "createdAt": {
                        "S": "2022-08-20"
                    },
                    "firstname": {
                        "S": "Sandro"
                    },
                    "ttl": {
                        "N": "1661166039"
                    },
                    "userId": {
                        "S": "user_2"
                    },
                    "lastname": {
                        "S": "Volpicella"
                    }
                },
                "SequenceNumber": "26163800000000026199463306",
                "SizeBytes": 104,
                "StreamViewType": "NEW_IMAGE"
            },
            "eventSourceARN": "arn:aws:dynamodb:eu-central-1:ACCOUNT:table/Users/stream/2022-08-21T15:34:25.720"
        }
    ]
}

Now you will see the event that is coming in. You will always get a Records array. Depending on your batch size more entries can be in there.

In the records, you can see your item in the dynamodb object. The most two important parts of that object are:

• Keys: These are the keys of the new item

• NewImage: The new user object

We can now use the NewImage and do something with it.

Considerations

Asynchronous Call

It is important to understand that your lambda will be called in an asynchronous fashion. That means you can use lambda destinations like onSuccess or onFailure to handle errors more gracefully and attach a Dead-Letter-Queue for example.

Idempotency

A DynamoDB stream call will be executed at least once. That means your lambda function can be invoked with the same event multiple times. You need to ensure that your call is idempotent. That means your operation is idempotent or you have a layer in-between to check that your function is only executed once. A good example can be seen in the AWS Lambda Powertools for Python.

Pricing

Streams are priced the following (US-east-1 example):

• The first 2.5 million DynamoDB Stream read requests are free

• After that each request costs $0.02 per 100k DynamoDB stream read units

Check out the DynamoDB pricing guide for more info.

Resources

Example implementation of Streams

Best practices and design patterns

Developer Guide for DynamoDB Streams

Final Words

That's it!

If you want to learn more about the fundamentals of AWS make sure to subscribe to our newsletter at awsfundamentals.com