AWS CloudWatch Logs: The Comprehensive Guide for Log Analysis and Insights
Let’s Take a Look at the Basic Concepts of AWS Cloudwatch
Table of contents
AWS CloudWatch is AWS's own logging, metrics, and alarm service. In this post, we will take a look at the centralized logging component of CloudWatch.
One of the core functionalities of CloudWatch is CloudWatch Logs.
This is the centralized logging space in AWS. Services like Lambda, API Gateway, or ECS are all logging directly into CloudWatch Logs.
This is a huge benefit when working with AWS. You have one central space where all your logs are stored.
Logs are text outputs of your application
Logs are the text output of your application. For example, if you put a print statement (console.log for node.js) into your Lambda function and run it. You will find the log in CloudWatch.
Often you will use a dedicated logger within your application that logs more than just a message. In the following example we have used a JSON logger to log an example message:
{
"message": "This is a message",
"awsRegion": "eu-central-1",
"functionName": "ThumbnailLambda",
"functionVersion": "$LATEST",
"functionMemorySize": "512",
"awsRequestId": "ghjkghjk-7466-497e-ac55-3d9c1d9beee0",
"x-correlation-id": "ghjkgkhj-7466-497e-ac55-3d9c1d9beee0",
"sLevel": "DEBUG"
}
CloudWatch follows the concepts of Log Streams, Groups, and Events
Let’s first dive into some concepts of CloudWatch
| Name | Definition |
| Log Events | A log event is your actual log statement. It contains the timestamp of your log and the raw log statement you put into it. |
| Log Streams | A log stream contains one or more log events from the same source. For example, a log stream of a Lambda function can contain more executions of the same Lambda. |
| Log Groups | A log group is a container that holds multiple log streams. Typically one log group is dedicated to one service. One Lambda function for example has one log group. |
| Metric Filter | You can create metric filters to get metrics out of your log events. |
| Retention Setting | The retention setting defines how long your logs are stored in CloudWatch. CloudWatch also costs money so it is important to not store the data indefinitely. |
Log Events are the actual text outputs of your application
Log events are the actual text output. That means everything you put on your standard output in your application will land in CloudWatch.
These events contain a Timestamp and the actual message. You can see the start and the end of this Lambda execution indicated by START and END.
Log Streams contain one or more log events
A log stream is a bucket for all log events.
In the case of Lambda, one log stream belongs to one warm Lambda container. This means one Lambda container that wasn’t destroyed (head over to the Lambda chapter if you want to understand this deeper). This log stream holds the log events.
Log Groups hold all logs for one application or service
Here are all the different Log Groups:
The name of a log group is prefixed with /aws and the service name. For Lambda this is: /aws/lambda/<FUNCTION_NAME>.
Final Words
These are the basic concepts of CloudWatch Logs. There are much more things to learn about CloudWatch like
Metrics & Alarms
Logs Insights
X-Ray
Synthetics
Evidently
You can have a full overview of CloudWatch here or in our book AWS Fundamentals - AWS for the Real World.
Related Reads
If you found this comprehensive guide on AWS CloudWatch Logs insightful, you might also enjoy these related posts: