> ## Documentation Index
> Fetch the complete documentation index at: https://docs.bronto.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom Log Ingestion

> Send log events directly to Bronto via HTTP using the Ingestion API — with full control over compression, batching, metadata, and authentication.

The Bronto Ingestion API lets you POST log events directly to Bronto from any HTTP client or script. This is useful when you need to build a custom pipeline, integrate with an existing system, or ingest logs programmatically without an agent.

## Prerequisites

You'll need an API key with ingestion or admin permissions. See [API Keys](/Account-Management/API-Keys) for how to create one.

## Endpoints

* `https://ingestion.eu.bronto.io` — EU region
* `https://ingestion.us.bronto.io` — US region

OTLP logs can also be sent to `https://ingestion.eu.brontobytes.io/v1/logs` or `https://ingestion.us.brontobytes.io/v1/logs`.

## Request Format

All requests must be `POST` with a JSON Lines (NDJSON) body — one JSON object per line.

### Required headers

| Header             | Value               |
| ------------------ | ------------------- |
| `x-bronto-api-key` | Your Bronto API key |
| `Content-Type`     | `application/json`  |

### Recommended headers

These headers control how your data is organized in Bronto. See [Data Organization](/Search-and-Visualize/Partitions) for how datasets, collections, and tags work.

| Header                | Description                              |
| --------------------- | ---------------------------------------- |
| `x-bronto-dataset`    | Dataset to ingest into                   |
| `x-bronto-collection` | Collection name                          |
| `x-bronto-tags`       | Comma-separated tags to attach to events |

### Other headers

| Header             | Description                                      |
| ------------------ | ------------------------------------------------ |
| `Content-Encoding` | Compression format: `gzip`, `zstd`, or `deflate` |

### Event schema

Each line in the request body should be a JSON object with the following fields:

| Field       | Required    | Description                                      |
| ----------- | ----------- | ------------------------------------------------ |
| `message`   | Yes         | The log content string                           |
| `timestamp` | Recommended | ISO 8601 timestamp (e.g. `2024-01-15T10:30:00Z`) |

## Send logs

### Uncompressed

```bash Send logs theme={"dark"}
curl -X POST https://ingestion.eu.bronto.io \
  -H "x-bronto-api-key: <YOUR_API_KEY>" \
  -H "Content-Type: application/json" \
  --data-binary '{"timestamp":"2024-01-15T10:30:00Z","message":"user login successful","user":"alice"}
{"timestamp":"2024-01-15T10:30:01Z","message":"request completed","status":200}'
```

### With gzip compression

```bash Send logs with gzip theme={"dark"}
echo '{"timestamp":"2024-01-15T10:30:00Z","message":"user login successful"}' | \
  gzip | \
  curl -X POST https://ingestion.eu.bronto.io \
    -H "x-bronto-api-key: <YOUR_API_KEY>" \
    -H "Content-Type: application/json" \
    -H "Content-Encoding: gzip" \
    --data-binary @-
```

### With Zstandard compression

```bash Send logs with zstd theme={"dark"}
echo '{"timestamp":"2024-01-15T10:30:00Z","message":"user login successful"}' | \
  zstd | \
  curl -X POST https://ingestion.eu.bronto.io \
    -H "x-bronto-api-key: <YOUR_API_KEY>" \
    -H "Content-Type: application/json" \
    -H "Content-Encoding: zstd" \
    --data-binary @-
```

### With deflate compression

```bash Send logs with deflate theme={"dark"}
echo '{"timestamp":"2024-01-15T10:30:00Z","message":"user login successful"}' | \
  python3 -c "import sys,zlib; sys.stdout.buffer.write(zlib.compress(sys.stdin.buffer.read()))" | \
  curl -X POST https://ingestion.eu.bronto.io \
    -H "x-bronto-api-key: <YOUR_API_KEY>" \
    -H "Content-Type: application/json" \
    -H "Content-Encoding: deflate" \
    --data-binary @-
```

## Batch processing

For large files, use a shell script to split the file into chunks and send each one:

```bash Batch ingest from file theme={"dark"}
#!/bin/bash
FILE="logs.ndjson"
BATCH_SIZE=5000
API_KEY="<YOUR_API_KEY>"
ENDPOINT="https://ingestion.eu.bronto.io"

split -l $BATCH_SIZE "$FILE" /tmp/batch_

for batch in /tmp/batch_*; do
  gzip -c "$batch" | curl -s -X POST "$ENDPOINT" \
    -H "x-bronto-api-key: $API_KEY" \
    -H "Content-Type: application/json" \
    -H "Content-Encoding: gzip" \
    --data-binary @-
  echo "Sent $batch"
done

rm /tmp/batch_*
```

## Compression ratios

Compressing payloads before sending reduces transfer size significantly. Typical compression ratios for CDN logs:

| Algorithm | `Content-Encoding` value | Typical ratio |
| --------- | ------------------------ | ------------- |
| gzip      | `gzip`                   | \~7.8%        |
| Zstandard | `zstd`                   | \~6.0%        |
| Deflate   | `deflate`                | \~7.8%        |

<Note>
  Zstandard offers the best compression ratio and is recommended for high-volume pipelines.
</Note>

## Payload size limits

| Payload type           | Maximum size             |
| ---------------------- | ------------------------ |
| Uncompressed           | 10 MB (10,485,760 bytes) |
| Compressed (wire size) | 10 MB (10,485,760 bytes) |

Exceeding these limits returns an HTTP `413` response.

## Response codes

| Code  | Meaning                           |
| ----- | --------------------------------- |
| `200` | Success — events ingested         |
| `400` | Malformed request or invalid JSON |
| `401` | Invalid or missing API key        |
| `413` | Payload exceeds size limit        |
| `429` | Rate limit or quota exhausted     |
| `500` | Server error                      |

## Reliable delivery

A `200` response means your data has been securely stored and will be ingested — no data will be lost. If the API returns an error response, no data from that request will be ingested.

<Note>
  Bronto is built for reliable, end-to-end delivery. Once you receive a `200`, your logs are safe.
</Note>