Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.bronto.io/llms.txt

Use this file to discover all available pages before exploring further.

Fluentd forwards logs to Bronto using its built-in http output plugin. This page covers Fluentd configuration only. For installation instructions, see the Fluentd installation guide. For the full Fluentd configuration reference, see the Fluentd documentation.

​
Endpoint and authentication

Use the ingestion endpoint for your Bronto region:
RegionEndpoint
EUhttps://ingestion.eu.bronto.io:443
UShttps://ingestion.us.bronto.io:443
Every request requires these headers:
HeaderRequiredDescription
x-bronto-api-keyRequiredYour Bronto API key.
x-bronto-datasetRequiredThe dataset to route logs to.
x-bronto-collectionRequiredThe collection to group datasets under.
x-bronto-tagsOptionalKey-value tag pairs (e.g. env=prod,team=platform). See Partitions.

​
Minimal configuration

Tail a JSON log file and forward to Bronto.
/etc/fluent/fluentd.conf
<source>
  @type tail
  path /path/to/your/logs
  tag app.logs
  refresh_interval 5s
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/fluentd.pos
</source>

<match app.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post

  <buffer>
    @type file
    path /var/log/td-agent/buffer/http
    flush_interval 10s
    chunk_limit_size 5MB
    overflow_action block
  </buffer>

  <format>
    @type json
  </format>

  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "<YOUR_DATASET_NAME>", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>
For the full HTTP output configuration reference, see the Fluentd HTTP output documentation.

​
Parsing unstructured logs

Rather than building Fluentd parsers for unstructured text, ship raw log lines to Bronto and use the Bronto Custom Parser to extract structured fields server-side. The Custom Parser uses LLMs to generate parsers automatically and ships with built-in support for Apache, IIS, HAProxy, Syslog, key-value, and custom formats — no regex maintenance required.

​
Common patterns

​
Adding metadata to every log

Use the record_transformer filter to inject metadata into every record. Values can be expressions (Ruby) or environment variables. 
<filter **>
  @type record_transformer
  <record>
    hostname "#{Socket.gethostname}"
    branch "#{ENV['APP_GIT_BRANCH']}"
  </record>
</filter>

​
Routing multiple log sources to separate datasets

Tag each <source> distinctly, then use a separate <match> block for each tag with the appropriate x-bronto-dataset header. 
<source>
  @type tail
  path /var/log/api.log
  tag api.logs
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/api.pos
</source>

<source>
  @type tail
  path /var/log/worker.log
  tag worker.logs
  <parse>
    @type json
  </parse>
  pos_file /var/log/td-agent/buffer/worker.pos
</source>

<match api.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post
  <format>
    @type json
  </format>
  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "api-service", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>

<match worker.logs>
  @type http
  endpoint https://ingestion.<REGION>.bronto.io:443
  http_method post
  <format>
    @type json
  </format>
  headers {"x-bronto-api-key": "<YOUR_API_KEY>", "x-bronto-dataset": "worker-service", "x-bronto-collection": "<YOUR_COLLECTION_NAME>"}
</match>

​
Verify log collection

Once you have applied your configuration and restarted Fluentd, you can expect to see your log data being ingested to Bronto and accessible via the Search page.

​
Further reading