> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bronto.io/llms.txt
> Use this file to discover all available pages before exploring further.
# Forwarding AWS Logs from CloudWatch
> Forward Amazon CloudWatch Logs to Bronto with Lambda subscription filters to centralize AWS service, application, and Lambda function logs for search.
## When to Use
The CloudWatch Log Forwarder is a good fit when:
* Your AWS services already deliver logs to CloudWatch (Lambda, RDS, ECS, API Gateway, VPC Flow Logs, and others)
* You want a simple setup without changing how your services log
* You are comfortable with the CloudWatch ingestion fees that apply to your log volume
If you want to avoid CloudWatch ingestion fees for new workloads, consider [ECS FireLens](./aws-firelens), [Fluent Bit on EKS](./aws-fluent-bit-eks), or [ADOT](./aws-adot) instead. See the [overview](./aws-overview) for a full cost and method comparison.
***
## Supported AWS Services
The CloudWatch Log Forwarder is the recommended path for any AWS service that publishes logs to CloudWatch Logs:
| Service | Log type |
| -------------------------------- | ------------------------------------------------------------------------------------------------- |
| AWS Lambda | Function `stdout` / `stderr` (for functions not using the [ADOT Lambda Layer](./aws-adot-lambda)) |
| Amazon RDS / Aurora (PostgreSQL) | Database engine and slow-query logs |
| Amazon RDS / Aurora (MySQL) | Error, slow-query, and general logs |
| Amazon API Gateway | Execution and access logs |
| Amazon ElastiCache | Engine and slow logs |
| Amazon Route 53 | DNS query logs |
| Amazon SES | Sending event logs |
| AWS Step Functions | State machine execution logs |
| Amazon Cognito | User pool authentication logs |
| VPC Flow Logs | Network flow records (CloudWatch delivery) |
| AWS WAF | Web ACL traffic logs (CloudWatch delivery) |
Any service that can deliver to a CloudWatch log group is supported. See the [AWS documentation on services that publish to CloudWatch Logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) for the full list.
***
## How it Works
The Bronto AWS Log Forwarder is built around an AWS Lambda function that processes and forwards logs to Bronto. The function is triggered through CloudWatch [subscription filters](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters.html). Both log group level and [account level](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters-AccountLevel.html) subscription filters are supported.
Bronto recommends using **account level subscription filters** so all log groups are forwarded automatically without per-group configuration.
***
## Setup Instructions
Choose your preferred deployment method:
**Terraform (recommended)** — Deploys the Lambda forwarder along with the necessary IAM roles and policies. Supports routing different log types to different destinations within Bronto. Full details and usage instructions are in the [Terraform module repository](https://github.com/brontoio/brontobytes-aws-ingestion-terraform).
**CloudFormation** — Use the [CloudFormation template](https://s3.eu-west-1.amazonaws.com/brontobytes.io/integration/aws/cloudformation/1.0.0/brontoForwarder.yaml) in the AWS Console to create the stack by filling out a form.
**Manual** — The Lambda function code and a pre-packaged deployment artifact are available in the [GitHub repository](https://github.com/brontoio/brontobytes-aws-ingestion-python).
Set up account level subscription filters to automatically forward all CloudWatch log groups to the deployed Lambda function. Refer to the [AWS documentation on subscription filters](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/SubscriptionFilters-AccountLevel.html) for setup steps.
***
## Data Organization
The forwarder Lambda controls how data lands in Bronto through its configuration, not via HTTP headers — it composes the `x-bronto-*` headers internally based on the log group name and the per-source mapping you provide. See [Data Organization](/Search-and-Visualize/Partitions) for how datasets, collections, and tags work in Bronto.
By default, with an account-level subscription filter (the recommended setup):
* **Dataset** is the CloudWatch log group name (e.g. `/aws/lambda/`).
* **Collection** is whatever you set as the default — `cloudwatch_default_collection` — or Bronto's default collection if not set.
* **Tags** come from the global `bronto_tags` map.
Provide a `destination_config` entry only when you want to override the defaults for a specific log group, or to apply per-source tags.
### Configure via Terraform (recommended)
Set defaults and any per-log-group overrides via the [Terraform module](https://github.com/brontoio/brontobytes-aws-ingestion-terraform#usage):
```hcl theme={"dark"}
module "bronto_aws_log_forwarding" {
source = "git::https://github.com/brontoio/brontobytes-aws-ingestion-terraform.git//aws_log_forwarder"
# ... bronto_api_key, artifact_bucket, etc.
cloudwatch_default_collection = "Cloudwatch"
account_level_cloudwatch_subscription = {
enable = true
excluded_log_groups = ["log_group1", "log_group2"]
}
# Optional — only needed to override defaults for specific log groups
destination_config = {
"/aws/lambda/" = {
dataset = ""
collection = ""
log_type = "cloudwatch_log"
tags = { team = "platform" }
}
}
bronto_tags = { environment = "production", region = "eu-west-1" }
}
```
* `cloudwatch_default_collection` sets the Bronto collection for any log group not explicitly listed in `destination_config`.
* `account_level_cloudwatch_subscription` is the recommended way to forward all log groups automatically. AWS allows only one account-level subscription, so this is only suitable if no other account-level subscription is already in place.
* `bronto_tags` apply globally; per-source `tags` inside a `destination_config` entry add or override tags for that log group only.
* For log groups that need their own subscription filter pattern, set `set_individual_subscription = true` and `subscription_filter_pattern = ""` inside the matching `destination_config` entry. A `default_subscription_filter_pattern` can be applied to every account-level subscription.
For the full variable reference, see the [Terraform module README](https://github.com/brontoio/brontobytes-aws-ingestion-terraform#usage).
### Configure direct Lambda deployments
If you deploy the [Python Lambda](https://github.com/brontoio/brontobytes-aws-ingestion-python/blob/main/README.md) directly (CloudFormation or manual), the same configuration is provided as Lambda environment variables:
| Environment variable | Purpose |
| ------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------- |
| `bronto_api_key` | Bronto API key |
| `bronto_endpoint` | Regional ingestion endpoint, e.g. `https://ingestion.eu.bronto.io/` |
| `cloudwatch_default_collection` | Default Bronto collection for CloudWatch log groups |
| `destination_config` | Base64-encoded JSON map of log group names to `dataset`, `collection`, `log_type: cloudwatch_log`, and optional `tags` — only needed for overrides |
| `tags` | Comma-separated `key=value` pairs applied to all datasets — e.g. `environment=production,region=eu-west-1` |
See the [Python forwarder README](https://github.com/brontoio/brontobytes-aws-ingestion-python/blob/main/README.md) for a sample `destination_config` payload and the full attribute reference.
***
## Cost Notes
* AWS charges for CloudWatch log ingestion (PUT requests and GB ingested to CloudWatch). These fees apply regardless of whether you forward the logs to Bronto.
* Lambda invocation costs for the forwarder itself are minimal.
* If CloudWatch ingestion costs are a concern at your log volume, see [ECS FireLens](./aws-firelens), [Fluent Bit on EKS](./aws-fluent-bit-eks), or [ADOT](./aws-adot) for alternatives that bypass CloudWatch entirely.
***
For assistance, contact [support@bronto.io](mailto:support@bronto.io).