Table of contents
CloudWatch and CloudTrail are two services that often get confused with each other. While both are doing completely different things, this post will show you what each service does and when to use which one.
Both services are monitoring and logging services. CloudWatch is responsible for monitoring your compute resources and AWS Services. CloudTrail is responsible for logging your internal AWS Usage. We will look into both services in more detail.
Let's have a quick summary first:
What is AWS CloudWatch?
Amazon CloudWatch is AWS’s central logging and metrics service. Each AWS service logs to CloudWatch. For example, your Lambda function, ECS cluster, or sagemaker instances can all log to CloudWatch.
CloudWatch consists of multiple different products. The most popular are:
CloudWatch Metrics: Metrics repository to store and analyze metrics
CloudWatch Alarms: Create automated alarms that react on the threshold of metrics
CloudWatch X-Ray: Trace distributed requests
CloudWatch Synthetics: Regular checks of the health of your web application
CloudWatch Evidently: A/B Testing & Feature Flagging
Use Cases for AWS CloudWatch
Everything that is related to your application logs or metrics are valid CloudWatch use cases. Some examples are:
Alarms for Lambda Errors
A common use case for CloudWatch is an automated alarm. You can configure your alarm for the following case: If Lambda XY throws 5 errors per minute create an alarm.
This alarm can have different actions such as:
Call another Lambda Function
CloudWatch also allows you to build Dashboards. With Dashboards it is easy to get an overview of how your system behaves and how huge your current workload is.
Dashboards can also be shared with other AWS Accounts or with other stakeholders. CloudWatch generates automatic Dashboards for the most popular services like DynamoDB, API Gateway, or Lambda.
Debug User Requests
Debugging user requests can get quite challenging in a distributed cloud environment. With CloudWatch you are able to find logs more easily by using CloudWatch Log Insights and X-Ray.
Log Insights allows you to query logs in a SQL-like language in multiple log groups. With X-Ray on the other hand you can dig into single-user traces and see which services the request went through. X-Ray also creates a service map for you:
Remember: CloudWatch is everything that is connected to your applications logs, health, or metrics.
You can get a more in-depth guide on CloudWatch here
What is AWS CloudTrail?
Now let's see what the service AWS CloudTrail is.
AWS CloudTrail provides a detailed record of all actions taken within an AWS account. This includes actions taken by users, services, and applications. CloudTrail records all API activity within an AWS account.
CloudTrail is often used for auditing mechanisms. For example, if you want to know which user changed a bucket's visibility setting. You can find this information in the CloudTrail service.
How to use CloudTrail?
CloudTrail is opposed to CloudWatch, not automatically activated. You need to go to the CloudTrail service and create a trail.
While CloudTrail doesn't automatically cost something, the CloudTrail logs are saved on S3. You will pay for the storage on S3 to store these logs. After creating the trail, all your management events will now be logged to an S3 bucket.
Each API activity will now be logged directly to an S3 bucket and can be further analyzed. There will be a large number of logs after some time. To analyze the logs more efficiently you need to use a service such as Athena for querying the logs from S3.
Security & Compliance
The main use case of CloudTrail is to monitor your internal usage of your AWS Account. CloudTrail lets you understand which user or application changed which resource in AWS. This is often necessary to meet compliance requirements like PCI, HIPAA, SOX.
CloudTrail logs can also be used if somebody was able to attack your AWS account or even gained access to it. With CloudTrail you can investigate security incidents and conduct a forensic analysis.
Troubleshooting & Incident Management
With CloudTrail you can investigate and troubleshoot incidents that occurred within AWS. Your AWS Account can be changed programitticaly and it is important to understand the root cause of incidents. CloudTrail logs help you with that.
Okay, we now know what CloudWatch and what CloudTrail is. When should we use which service now?
When to use CloudWatch?
You always use CloudWatch if you are interested in the inner workings of your application. This can be the health of your application, the number of errors, or also detailed log statements.
When to use CloudTrail?
CloudTrail is always used if you want to know which user or resource did which API action. Examples of that are:
Who created the S3 Bucket
Who changed the concurrency setting of a Lambda Function
Who shut down the RDS Server
This is especially important for having an automatic audit log of all of your actions. Since Cloud development can be highly automated this is also useful for understanding how your applications interact with each other.
In conclusion, while both AWS CloudWatch and CloudTrail are monitoring and logging services, they serve different purposes. CloudWatch is used to monitor and log the internal workings of an application, while CloudTrail is used to track all API activities that take place within an AWS account.
CloudWatch provides various products, such as Logs, Metrics, Alarms, X-Ray, Synthetics, and Evidently. These products can be used to set alarms for Lambda errors, create dashboards, and debug user requests.
On the other hand, CloudTrail records all API activities in an AWS account and is often used for auditing mechanisms, such as tracking which user changed a bucket's visibility setting. CloudTrail needs to be manually activated, and the logs are stored on S3, which means that there are additional storage costs.
In summary, it's essential to understand the differences between CloudWatch and CloudTrail to make informed decisions about which service to use. CloudWatch is ideal for monitoring and logging the internal workings of an application, while CloudTrail is perfect for tracking all API activities in an AWS account, which is critical for auditing and compliance purposes.