Overview
Bronto’s Kinesis Firehose integration allows you to stream data directly from an AWS Kinesis Firehose delivery stream into Bronto. This is well suited to high-throughput pipelines where data already flows through Kinesis — for example, VPC Flow Logs, CloudTrail events, or custom application events published to a Kinesis stream. The integration requires no Lambda function. Firehose delivers data directly to Bronto’s HTTP endpoint, handling batching and retries natively.When to Use Firehose
| Situation | Recommendation |
|---|---|
| Data already in a Kinesis stream | Firehose is the natural choice |
| Very high event volume (GB/hour+) | Firehose scales without additional Lambda cost |
| You want managed delivery with built-in retries | Firehose handles this natively |
| Application or container logs | Use ADOT or the CloudWatch Forwarder instead |
Supported AWS Services
Kinesis Firehose is the recommended path for high-volume sources that already flow through Kinesis, and for services that deliver directly to a Firehose stream. Many CloudWatch-native services can also be routed to Firehose via a CloudWatch Logs subscription filter when CloudWatch ingestion costs become significant.| Service | Log type |
|---|---|
| Amazon CloudFront (real-time logs) | Per-request edge logs (native Firehose delivery) |
| VPC Flow Logs | Network flow records (high-volume deployments) |
| AWS CloudTrail | Management and data events (high-volume deployments) |
| AWS WAF | Web ACL traffic logs (native Firehose delivery) |
| Amazon API Gateway | Access logs (high-volume APIs, via CloudWatch subscription) |
| AWS Lambda | Function logs (high-volume functions, via CloudWatch subscription) |
| Amazon RDS / Aurora | Database engine logs (high-volume, via CloudWatch subscription) |
| Amazon ElastiCache | Engine and slow logs (high-volume, via CloudWatch subscription) |
| Amazon Route 53 | DNS query logs (high-volume, via CloudWatch subscription) |
| Amazon SES | Sending event logs (high-volume) |
| Custom Kinesis pipelines | Any application events published to a Kinesis Data Stream or Firehose |
Data Organization
Bronto’s Firehose preview supports the standard recommended headers for routing data into the right place — see Data Organization for how datasets, collections, and tags work.| Header | Description |
|---|---|
x-bronto-dataset | Dataset to ingest into |
x-bronto-collection | Collection name |
x-bronto-tags | Comma-separated tags to attach to events |