Skip to main content

​
When to Use

The CloudWatch Log Forwarder is a good fit when:
  • Your AWS services already deliver logs to CloudWatch (Lambda, RDS, ECS, API Gateway, VPC Flow Logs, and others)
  • You want a simple setup without changing how your services log
  • You are comfortable with the CloudWatch ingestion fees that apply to your log volume
If you want to avoid CloudWatch ingestion fees for new workloads, consider ECS FireLens, Fluent Bit on EKS, or ADOT instead. See the overview for a full cost and method comparison.

​
Supported AWS Services

The CloudWatch Log Forwarder is the recommended path for any AWS service that publishes logs to CloudWatch Logs:
ServiceLog type
AWS LambdaFunction stdout / stderr (for functions not using the ADOT Lambda Layer)
Amazon RDS / Aurora (PostgreSQL)Database engine and slow-query logs
Amazon RDS / Aurora (MySQL)Error, slow-query, and general logs
Amazon API GatewayExecution and access logs
Amazon ElastiCacheEngine and slow logs
Amazon Route 53DNS query logs
Amazon SESSending event logs
AWS Step FunctionsState machine execution logs
Amazon CognitoUser pool authentication logs
VPC Flow LogsNetwork flow records (CloudWatch delivery)
AWS WAFWeb ACL traffic logs (CloudWatch delivery)
Any service that can deliver to a CloudWatch log group is supported. See the AWS documentation on services that publish to CloudWatch Logs for the full list.

​
How it Works

The Bronto AWS Log Forwarder is built around an AWS Lambda function that processes and forwards logs to Bronto. The function is triggered through CloudWatch subscription filters. Both log group level and account level subscription filters are supported. Bronto recommends using account level subscription filters so all log groups are forwarded automatically without per-group configuration.

​
Setup Instructions

1

Deploy the forwarder

Choose your preferred deployment method:Terraform (recommended) — Deploys the Lambda forwarder along with the necessary IAM roles and policies. Supports routing different log types to different destinations within Bronto. Full details and usage instructions are in the Terraform module repository.CloudFormation — Use the CloudFormation template in the AWS Console to create the stack by filling out a form.Manual — The Lambda function code and a pre-packaged deployment artifact are available in the GitHub repository.
2

Configure CloudWatch subscription filters

Set up account level subscription filters to automatically forward all CloudWatch log groups to the deployed Lambda function. Refer to the AWS documentation on subscription filters for setup steps.

​
Data Organization

The forwarder Lambda controls how data lands in Bronto through its configuration, not via HTTP headers — it composes the x-bronto-* headers internally based on the log group name and the per-source mapping you provide. See Data Organization for how datasets, collections, and tags work in Bronto. By default, with an account-level subscription filter (the recommended setup):
  • Dataset is the CloudWatch log group name (e.g. /aws/lambda/<function-name>).
  • Collection is whatever you set as the default — cloudwatch_default_collection — or Bronto’s default collection if not set.
  • Tags come from the global bronto_tags map.
Provide a destination_config entry only when you want to override the defaults for a specific log group, or to apply per-source tags. Set defaults and any per-log-group overrides via the Terraform module: 
module "bronto_aws_log_forwarding" {
  source = "git::https://github.com/brontoio/brontobytes-aws-ingestion-terraform.git//aws_log_forwarder"
  # ... bronto_api_key, artifact_bucket, etc.

  cloudwatch_default_collection = "Cloudwatch"

  account_level_cloudwatch_subscription = {
    enable              = true
    excluded_log_groups = ["log_group1", "log_group2"]
  }

  # Optional — only needed to override defaults for specific log groups
  destination_config = {
    "/aws/lambda/<function-name>" = {
      dataset    = "<YOUR_DATASET>"
      collection = "<YOUR_COLLECTION>"
      log_type   = "cloudwatch_log"
      tags       = { team = "platform" }
    }
  }

  bronto_tags = { environment = "production", region = "eu-west-1" }
}
  • cloudwatch_default_collection sets the Bronto collection for any log group not explicitly listed in destination_config.
  • account_level_cloudwatch_subscription is the recommended way to forward all log groups automatically. AWS allows only one account-level subscription, so this is only suitable if no other account-level subscription is already in place.
  • bronto_tags apply globally; per-source tags inside a destination_config entry add or override tags for that log group only.
  • For log groups that need their own subscription filter pattern, set set_individual_subscription = true and subscription_filter_pattern = "<pattern>" inside the matching destination_config entry. A default_subscription_filter_pattern can be applied to every account-level subscription.
For the full variable reference, see the Terraform module README.

​
Configure direct Lambda deployments

If you deploy the Python Lambda directly (CloudFormation or manual), the same configuration is provided as Lambda environment variables:
Environment variablePurpose
bronto_api_keyBronto API key
bronto_endpointRegional ingestion endpoint, e.g. https://ingestion.eu.bronto.io/
cloudwatch_default_collectionDefault Bronto collection for CloudWatch log groups
destination_configBase64-encoded JSON map of log group names to dataset, collection, log_type: cloudwatch_log, and optional tags — only needed for overrides
tagsComma-separated key=value pairs applied to all datasets — e.g. environment=production,region=eu-west-1
See the Python forwarder README for a sample destination_config payload and the full attribute reference.

​
Cost Notes

  • AWS charges for CloudWatch log ingestion (PUT requests and GB ingested to CloudWatch). These fees apply regardless of whether you forward the logs to Bronto.
  • Lambda invocation costs for the forwarder itself are minimal.
  • If CloudWatch ingestion costs are a concern at your log volume, see ECS FireLens, Fluent Bit on EKS, or ADOT for alternatives that bypass CloudWatch entirely.
For assistance, contact support@bronto.io.