The Limitless CloudFormation Stack with Lambda-backed Resources

The Limitless CloudFormation Stack with Lambda-backed Resources

Create and manage any AWS resource within your stack with the help of AWS Lambda even if they are not supported by CloudFormation yet.

Tobias Schmidt's photo
Tobias Schmidt
·Sep 14, 2022·

4 min read

Table of contents

  • Being Limitless
  • Integrating it

Just recently, I started experimenting with AWS Chatbot and its capabilities. Integration was pretty simple and I quickly managed to track my pipelines. The next thing I wanted to achieve is getting Slack notifications if one of my serverless application’s regions is unavailable because that’s something that requires immediate action.

So I needed to create a CloudWatch alarm that monitors my Route53 health checks which have actions to send SNS notifications if the region’s health changes.

As I did that, I intentionally bricked one of my regions just to notice that the CloudWatch alert simply did nothing —the metric at all didn’t even show any gathered metrics. StackOverflow quickly helped me to identify that I needed to create my alarm in us-east-1, because that’s the only region that is able to access Route53 metrics.

As I’m using the Serverless Framework and set up my Route53 health checks via CloudFormation resources, I was stuck now, because CloudFormation is not able to create resources in another region. Yeah, I could just do this with Terraform, but I was also sure, that there must be a way to get this integrated into my Serverless-managed CloudFormation stack. I’m not giving up easily.

There we go: AWS Lambda-backed custom CloudFormation resources!

Me, finding out about Lambda-backed CloudFormation resources

Being Limitless

Reading through the documentation, for me it felt like finding some holy grail. This basically frees you from all of CloudFormation’s limits like maximum template size or the fact that you can’t create resources in another region.

The quintessence: Lambda functions which are creating your custom resources in behalf of CloudFormation. That resolves to: you can create any resource, anywhere because you’ve got full access to the AWS API.

How does it work?

  • CloudFormation triggers your Lambda function with parameters you define and a specific event (Create, Update, or Delete) and a signed S3 URL.
  • Your Lambda function creates (or updates or deletes) your custom resource(s).
  • You notify CloudFormation about the successful (or unsuccessful) resource operation(s) by posting your results to the S3 URL.

Example for Lambda-backed resource creation with CloudFormation

Integrating it

Let’s recall our initial target: creating a CloudWatch Alarm in us-east-1, even though the stack resides in eu-central-1.

Let’s add the infra for our Lambda function that manages our Alarm:

functions:
  # [...]
  route53Alarms:
    package:
      include:
        - create-route-53-alarms.js
    handler: create-route-53-alarms.handler
    name: create-route-53-alarms
    reservedConcurrency: 1
    runtime: nodejs12.x
    memorySize: 1024
    timeout: 120

Then we add the custom CloudFormation resource:

resources:
  Resources:
    StatusHealthCheckAlarm:
      Type: 'Custom::LambdaBackedRoute53Alert'
      Properties:
        ServiceToken: !GetAtt Route53AlarmsLambdaFunction.Arn
        topicArn: 
        healthCheckId: !Ref RegionHealthCheck

In my example, I’m passing the identifier of the Route53 health check I want to track via the Alarm. The ServiceToken has to be the ARN of the Lambda function that should be invoked for managing the resource. We can reference this via !GettAtt.

And that’s all for the infrastructure side already.

The last step is to write the actual code for our function.

💡 There’s a dependency for Lambda-backed resources with Python, where you only need to define methods for all event types and annotate them accordingly. As I love working with Node.js and I wanted to really understand what's going on, I didn’t make use of this.

const AWS   = require('aws-sdk')
const https = require('https')
const url   = require('url')

const cloudwatch = new AWS.CloudWatch({ region: 'us-east-1' })

exports.handler = async (event, context) => {
  try {
    switch (event.RequestType) {
      case 'Create': await createHealthCheck(event); break
      case 'Update': await updateHealthCheck(event); break
      case 'Delete': await deleteHealthCheck(event);
    }
    await sendResponse(event, context, 'SUCCESS')
  } catch (e) {
    console.log(e)
    await sendResponse(event, context, 'FAILURE')
  }
}

async function createHealthCheck(event) {
  await cloudwatch.putMetricAlarm({
    AlarmName: `region-unavailable`,
    ActionsEnabled: true,
    AlarmActions: [event.ResourceProperties.topicArn],
    OKActions: [event.ResourceProperties.topicArn],
    ComparisonOperator: 'LessThanThreshold',
    EvaluationPeriods: 1,
    MetricName: `HealthCheckStatus`,
    Namespace: 'AWS/Route53',
    Period: 60,
    Statistic: 'Minimum',
    Threshold: 1.0,
    Dimensions: [{
      Name: 'HealthCheckId',
      Value: event.ResourceProperties.healthCheckId,
    }]
  }).promise()
}

async function updateHealthCheck(event) {
  await deleteHealthCheck(event)
  await createHealthCheck(event)
}

async function deleteHealthCheck(event) {
  await cloudwatch.deleteAlarms({
    AlarmNames: [`region-unavailable}`]
  }).promise()
}

const sendResponse = (event, context, responseStatus) => {
  return new Promise((resolve, reject) => {
    const responseBody = JSON.stringify({
      Status: responseStatus,
      PhysicalResourceId: `${event.StackId}-${event.LogicalResourceId}`,
      StackId: event.StackId,
      RequestId: event.RequestId,
      LogicalResourceId: event.LogicalResourceId,
      Data: {},
    })

    const parsedUrl = url.parse(event.ResponseURL)
    const options = {
      hostname: parsedUrl.hostname,
      port: 443,
      path: parsedUrl.path,
      method: 'PUT',
      headers: {
        'content-type': '',
        'content-length': responseBody.length,
      },
    }
    const request = https.request(options, function(response) {
      console.log(response.statusCode)
      console.log(JSON.stringify(response.headers))
      context.done()
      resolve()
    })
    request.on('error', function(error) {
      console.error(error)
      context.done()
      reject()
    })
    request.write(responseBody)
    request.end()
  })
}

And that’s all we have to do. The next deployment via Serverless will trigger the CloudFormation stack update and therefore our function with the event type Create. If the CloudWatch Alarm can be successfully created, we’re passing the resource details to S3 including the SUCCESS status, which will be picked up by CloudFormation.

⚠️ Be sure to always submit a response to the signed S3 URL! CloudFormation has a really long timeout for waiting until you’ve submitted your results of the custom resource management. That’s why I’m always putting everything into a try/catch clause. If something goes horribly wrong, at least the status update with FAILURE will be submitted. Else you have to wait for the timeout of CloudFormation which is 1 hour!

If you’re changing any parameters for the custom resource, CloudFormation will notice and send your Lambda function the Update event. And finally — when you’re deleting your stack — CloudFormation will send the Delete event.

 
Share this