AWS CloudWatch Logs - An Introduction

AWS CloudWatch is AWS's own logging, metrics, and alarm service. In this post, we will take a look at the centralized logging component of CloudWatch.

One of the core functionalities of CloudWatch is CloudWatch Logs.

AWS CloudWatch Logs Overview

This is the centralized logging space in AWS. Services like Lambda, API Gateway, or ECS are all logging directly into CloudWatch Logs.

This is a huge benefit when working with AWS. You have one central space where all your logs are stored.

Logs are text outputs of your application

Logs are the text output of your application. For example, if you put a print statement (console.log for node.js) into your Lambda function and run it. You will find the log in CloudWatch.

Often you will use a dedicated logger within your application that logs more than just a message. In the following example we have used a JSON logger to log an example message:

{
    "message": "This is a message",
    "awsRegion": "eu-central-1",
    "functionName": "ThumbnailLambda",
    "functionVersion": "$LATEST",
    "functionMemorySize": "512",
    "awsRequestId": "ghjkghjk-7466-497e-ac55-3d9c1d9beee0",
    "x-correlation-id": "ghjkgkhj-7466-497e-ac55-3d9c1d9beee0",
    "sLevel": "DEBUG"
}

CloudWatch follows the concepts of Log Streams, Groups, and Events

Let’s first dive into some concepts of CloudWatch

Name	Definition
Log Events	A log event is your actual log statement. It contains the timestamp of your log and the raw log statement you put into it.
Log Streams	A log stream contains one or more log events from the same source. For example, a log stream of a Lambda function can contain more executions of the same Lambda.
Log Groups	A log group is a container that holds multiple log streams. Typically one log group is dedicated to one service. One Lambda function for example has one log group.
Metric Filter	You can create metric filters to get metrics out of your log events.
Retention Setting	The retention setting defines how long your logs are stored in CloudWatch. CloudWatch also costs money so it is important to not store the data indefinitely.

Log Events are the actual text outputs of your application

Log events are the actual text output. That means everything you put on your standard output in your application will land in CloudWatch.

AWS CloudWatch Logs - Log Event

These events contain a Timestamp and the actual message. You can see the start and the end of this Lambda execution indicated by START and END.

Log Streams contain one or more log events

A log stream is a bucket for all log events.

AWS CloudWatch Logs - Log Stream

In the case of Lambda, one log stream belongs to one warm Lambda container. This means one Lambda container that wasn’t destroyed (head over to the Lambda chapter if you want to understand this deeper). This log stream holds the log events.

Log Groups hold all logs for one application or service

Here are all the different Log Groups:

AWS CloudWatch Logs - Log Groups

The name of a log group is prefixed with /aws and the service name. For Lambda this is: /aws/lambda/<FUNCTION_NAME>.

Final Words

These are the basic concepts of CloudWatch Logs. There are much more things to learn about CloudWatch like

Metrics & Alarms
Logs Insights
X-Ray
Synthetics
Evidently

You can have a full overview of CloudWatch here or in our book AWS Fundamentals - AWS for the Real World.

AWS CloudWatch Logs: The Comprehensive Guide for Log Analysis and Insights

Let’s Take a Look at the Basic Concepts of AWS Cloudwatch

Table of contents