> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bronto.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Forwarding AWS Logs from S3

> Forward logs delivered to Amazon S3 buckets into Bronto using a Lambda-based forwarder for VPC flow, ALB, CloudFront, and other AWS service logs.

## When to Use

The S3 Log Forwarder is a good fit when:

* AWS services deliver log files directly to an S3 bucket (CloudFront, ALB, NLB, VPC Flow Logs, and others)
* You want a low-complexity setup with no CloudWatch ingestion fees
* You prefer Terraform or CloudFormation for infrastructure deployment

For application logs or traces from running workloads, see [ECS FireLens](./aws-firelens), [ADOT](./aws-adot), or the [overview](./aws-overview) for a full comparison.

***

## Supported AWS Services

The S3 Log Forwarder is the recommended path for AWS services that deliver logs directly to S3:

| Service                         | Log type                            |
| ------------------------------- | ----------------------------------- |
| Amazon CloudFront               | Standard access logs                |
| Application Load Balancer (ALB) | Access logs                         |
| Network Load Balancer (NLB)     | Access logs                         |
| VPC Flow Logs                   | Network flow records (S3 delivery)  |
| AWS WAF                         | Web ACL traffic logs                |
| AWS CloudTrail                  | Management and data event log files |
| AWS GuardDuty                   | Findings export                     |
| Amazon S3                       | Server access logs                  |

Any other AWS service that delivers logs to an S3 bucket can use this forwarder. See the [AWS documentation on services that publish to S3](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) for the full list.

***

## How it Works

The Bronto AWS Log Forwarder is built around an AWS Lambda function that processes and forwards logs to Bronto. The function is triggered through S3 or EventBridge event notifications when new objects are written to your bucket.

Supported AWS log sources include AWS CloudFront, Application Load Balancer (ALB), Network Load Balancer (NLB), VPC Flow Logs, and others. See the [AWS documentation](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html) for the full list of services that deliver to S3.

***

## Setup Instructions

<Steps>
  <Step title="Deploy the forwarder">
    Choose your preferred deployment method:

    **Terraform (recommended)** — Deploys the Lambda forwarder along with the necessary IAM roles and policies. Supports routing different log types to different destinations within Bronto. Full details and usage instructions are in the [Terraform module repository](https://github.com/brontoio/brontobytes-aws-ingestion-terraform).

    **CloudFormation** — Use the [CloudFormation template](https://s3.eu-west-1.amazonaws.com/brontobytes.io/integration/aws/cloudformation/1.0.0/brontoForwarder.yaml) in the AWS Console to create the stack by filling out a form.

    **Manual** — The Lambda function code and a pre-packaged deployment artifact are available in the [GitHub repository](https://github.com/brontoio/brontobytes-aws-ingestion-python).
  </Step>

  <Step title="Configure S3 event notifications">
    Configure your S3 bucket to trigger the Lambda function when new objects are created. Refer to the [AWS documentation on S3 notifications](https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-event-notifications.html) for setup steps.
  </Step>
</Steps>

***

## Data Organization

The forwarder Lambda controls how data lands in Bronto through its configuration, not via HTTP headers — it composes the `x-bronto-*` headers internally based on the per-source mapping you provide. See [Data Organization](/Search-and-Visualize/Partitions) for how datasets, collections, and tags work in Bronto.

### Configure via Terraform (recommended)

Map each S3 source (bucket name or CloudFront distribution ID) to its target dataset, collection, and log type using the `destination_config` variable on the [Terraform module](https://github.com/brontoio/brontobytes-aws-ingestion-terraform#usage):

```hcl theme={"dark"}
module "bronto_aws_log_forwarding" {
  source = "git::https://github.com/brontoio/brontobytes-aws-ingestion-terraform.git//aws_log_forwarder"
  # ... bronto_api_key, artifact_bucket, logging_bucket, etc.

  destination_config = {
    "<S3_BUCKET_NAME>" = {
      dataset    = "<YOUR_DATASET>"
      collection = "<YOUR_COLLECTION>"
      log_type   = "s3_access_log"
    }
    "<CLOUDFRONT_DISTRIBUTION_ID>" = {
      dataset    = "<YOUR_DATASET>"
      collection = "<YOUR_COLLECTION>"
      log_type   = "cf_standard_access_log"
    }
  }

  bronto_tags = { environment = "production", region = "eu-west-1" }
}
```

* `dataset` / `collection` route each source's logs to a specific Bronto destination.
* `log_type` tells the forwarder which parser to apply — supported values include `s3_access_log`, `alb_access_log`, `nlb_access_log`, `clb_access_log`, `vpc_flow_log`, `cloudtrail`, `cf_standard_access_log`, `cf_realtime_access_log`, `bedrock_s3`, and `default` (no parsing, for already-structured JSON).
* `bronto_tags` apply globally to all datasets produced by this forwarder. A per-source `tags` map inside a `destination_config` entry adds or overrides tags for that source only.
* For sources whose S3 object keys don't match the default AWS naming patterns (for example, logs moved between buckets), use `paths_regex` with a `dest_config_id` capture group to map them to `destination_config` entries.

For the full variable reference, see the [Terraform module README](https://github.com/brontoio/brontobytes-aws-ingestion-terraform#usage).

### Configure direct Lambda deployments

If you deploy the [Python Lambda](https://github.com/brontoio/brontobytes-aws-ingestion-python/blob/main/README.md) directly (CloudFormation or manual), the same configuration is provided as Lambda environment variables:

| Environment variable | Purpose                                                                                                    |
| -------------------- | ---------------------------------------------------------------------------------------------------------- |
| `bronto_api_key`     | Bronto API key                                                                                             |
| `bronto_endpoint`    | Regional ingestion endpoint, e.g. `https://ingestion.eu.bronto.io/`                                        |
| `destination_config` | Base64-encoded JSON mapping S3 resource names to `dataset`, `collection`, `log_type`, and optional `tags`  |
| `tags`               | Comma-separated `key=value` pairs applied to all datasets — e.g. `environment=production,region=eu-west-1` |
| `paths_regex`        | Base64-encoded list of regex patterns for non-standard S3 key layouts                                      |

See the [Python forwarder README](https://github.com/brontoio/brontobytes-aws-ingestion-python/blob/main/README.md) for a sample `destination_config` payload and the full attribute reference.

***

## Cost Notes

* No CloudWatch ingestion fees — logs are read directly from S3 by the Lambda function.
* You pay for S3 storage and Lambda invocations, both of which are minimal for typical log volumes.

***

For assistance, contact [support@bronto.io](mailto:support@bronto.io).