Connect Fluent Bit to Bronto
Fluent Bit must be installed on the server hosting your logs to ensure efficient and reliable log transportation.
Use the following single line command to configure the Fluent Bit repository and install the Fluent Bit log collector.
Manual instructions
Add Server GPG Key
Add Repo
In order to download the latest version of Fluent Bit, add the Fluent Bit repository to your source list.
Install Package
Install the Fluent Bit from the repository setup in the previous step
Configure Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Required | Specifies the name of the service. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to logs. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify Log Collection
Once you have applied your configuration and restarted Fluent Bit, you can expect to see your log data being ingested to Bronto and accessible via the Search page.
Use the following single line command to configure the Fluent Bit repository and install the Fluent Bit log collector.
Manual instructions
Add Server GPG Key
Add Repo
In order to download the latest version of Fluent Bit, add the Fluent Bit repository to your source list.
Install Package
Install the Fluent Bit from the repository setup in the previous step
Configure Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Required | Specifies the name of the service. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to logs. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify Log Collection
Once you have applied your configuration and restarted Fluent Bit, you can expect to see your log data being ingested to Bronto and accessible via the Search page.
Use the following single line command to configure the Fluent Bit repository and install the Fluent Bit log collector.
Manual instructions
Add Repo
In order to download the latest version of Fluent Bit, add the Fluent Bit repository to your yum repo list.
Install Package
Install the Fluent Bit from the repository setup in the previous step.
Configure Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Required | Specifies the name of the service. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to logs. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify Log Collection
Once you have applied your configuration and restarted Fluent Bit, you can expect to see your log data being ingested to Bronto and accessible via the Search page.
To retrieve the latest Fluent Bit docker image use the following command.
Configure Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Required | Specifies the name of the service. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to logs. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify Log Collection
Once you have applied your configuration and restarted Fluent Bit, you c
Configurate Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Required | Specifies the name of the service. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to logs. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify log collection
Once you have applied your configuration and restarted Fluent Bit, you can expect to see your log data being ingested to Bronto and accessible via the Search page.
To make Fluent Bit available on every node, it should be installed as a DaemonSet. The recommended way to deploy Fluent Bit is with the official Helm Chart: https://github.com/fluent/helm-charts
To add the Fluent Helm Charts repo use the following command
To validate that the repo was added you can run helm search repo fluent
to ensure the charts were added. The default chart can then be installed by running the following
Configure Fluent Bit
After installation, Fluent Bit requires an output configuration to communicate with the Bronto system.
Parameter | Value | Requirement | Description | ||||
---|---|---|---|---|---|---|---|
name | HTTP | Required | Specifies the output plugin used by Fluent Bit. | ||||
tls | on | Required | Enables the use of TLS for secure communication. | ||||
host | ingestion<REGION>.bronto.io | Required | The Bronto endpoint responsible for ingesting log data. Supported regions: EU & US. | ||||
port | 443 | Required | Specifies the network port for communication. | ||||
format | json_lines or msgpack | Required | Specify the data format to be used in the HTTP request body, by default it uses msgpack. | ||||
compress | gzip | Optional | Recommended. Compresses the payload in GZIP format. Bronto supports and recommends using gzip for efficient data transmission. | ||||
header (x-bronto-service-name) | Your Service Name | Optional | Specifies the name of the service. Setting this header in a Kubernetes setup is not recommended. | ||||
header (x-bronto-service-namespace) | Your Service Namespace | Optional | Specifies the namespace for organizational purposes. Setting this header in a Kubernetes setup is recommended, with its value indicating the cluster from which the data is originating, e.g. clusterA-production-eu1 | ||||
header (x-bronto-tags) | Tags | Optional | Specifies the tags to apply to all logs of the cluster. A tag is a key-value pair applied to a log to hold metadata about that log. Each tag is a label consisting of a key and a required value. |
The table below lists the parameters used to determine the Bronto dataset associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-name | HTTP Header |
x-bronto-log-name | HTTP Header |
kubernetes.annotations.logs.bronto.io/dataset | Log Metadata |
service.name | Log Metadata |
service_name | Log Metadata |
kubernetes.annotations.service_name | Log Metadata |
kubernetes.labels.app.kubernetes.io/name | Log Metadata |
kubernetes.container_name | Log Metadata |
The table below lists the parameters used to determine the Bronto collection associated with the ingested data, ordered from highest to lowest precedence.
Parameter Name | Parameter Type |
---|---|
x-bronto-service-namespace | HTTP Header |
x-bronto-logset | HTTP Header |
kubernetes.annotations.logs.bronto.io/collection | Log Metadata |
service.namespace | Log Metadata |
service_namespace | Log Metadata |
Finally the table below lists the parameters used to assign tags to datasets, ordered from highest to lowest precedence.
Parameter Name | Parameter Type | Example |
---|---|---|
x-bronto-tags | HTTP Header | x-bronto-tags: tag1=value1,tag2=value2,… |
kubernetes.annotations.tags.logs.bronto.io/<TAG_NAME> | Log Metadata | kubernetes.annotations.tags.logs.bronto.io/tag1: value1 |
Verify log collection
Once you have applied your configuration and restarted Fluent Bit, you can expect to see your log data being ingested to Bronto and accessible via the Search page.