---
title: "Fluent Operator in Kubernetes | Hostman Docs"
description: "Learn how Fluent Operator simplifies Kubernetes log management by automating Fluent Bit and Fluentd deployment, configuration, and scaling with CRDs. Integrate easily with Elasticsearch, Loki, Kafka, and more."
---

> For the complete documentation index for AI agents, see [llms.txt](https://hostman.com/llms.txt).

In Kubernetes clusters, applications and system services generate a large number of logs. To facilitate analysis and storage of these logs, they can be collected in systems such as Elasticsearch, Loki, Kafka, Prometheus, and others.

Fluent Bit and Fluentd are tools for log collection and processing. However, configuring them in Kubernetes can be complex:

-   Manually creating resources (DaemonSet, StatefulSet) is required.
-   Configuration files need to be set up.
-   Container runtime specifics must be considered.
-   Secrets must be managed for authentication.
-   Updates and compatibility need to be monitored.

Fluent Operator simplifies this process by:

-   Automating the deployment of Fluent Bit and Fluentd, creating the necessary resources.
-   Using Custom Resource Definitions (CRD) to configure inputs, filters, outputs, and other parameters via YAML manifests.
-   Managing updates and scaling of Fluent Bit/Fluentd in response to cluster changes.
-   Simplifying integration with Elasticsearch, Loki, Kafka, Prometheus, and others.

Instead of manually configuring Fluent Bit and Fluentd, you install Fluent Operator and describe the settings. The operator brings the cluster to the desired state and automatically updates the agents when the configuration changes.

**Main Components**

-   **Fluent Bit**: A lightweight log collection agent.
-   **Fluentd**: A powerful agent that can be used as a DaemonSet or StatefulSet for advanced filtering and buffering.
-   **Fluent Operator**: Manages Fluent Bit and Fluentd through CRDs.

## Installation

You can install Fluent Operator via the Hostman dashboard:

1.  Go to the **Kubernetes** section and click on the cluster.
2.  Navigate to the **Addons** tab and select **Fluent Operator**.
3.  Enable **Advanced setup** and adjust the configuration for your needs.
4.  Click **Install**.

## Installation Parameters

### Operator

The `operator` section:

-   `containerRuntime`: Defines the container runtime used. In our managed Kubernetes, containerd is used. This value does not need to be changed.
-   `logPath`: Log file path. For containerd, it is `/var/log`.
-   `disableComponentControllers`: A list of Fluent Operator components to disable. Possible values: `fluent-bit` and `fluentd`. By default, both components are installed.

### Fluent Bit

The `input` section:

-   **tail**:
    -   `enable: true` — Enables log collection from files located at `/var/log/containers/*.log`.
    -   `path: "/var/log/containers/*.log"` — Defines the path to container log files.
    -   `refreshIntervalSeconds: 10` — The frequency (in seconds) for refreshing the log file list.
    -   `memBufLimit: 100MB` — Memory limit for buffering.
    -   `storageType: memory` — Storage type for the buffer. An alternative is filesystem, but a volume must be prepared in advance.
    -   `readFromHead: false` — Starts reading logs from the end.
    -   `skipLongLines: true` — Skips excessively long lines.
-   **systemd**:
    -   `enable: true` — Enables log collection from `journald`.
    -   `includeKubelet: true` — Includes kubelet logs from `journald`.
    -   `storageType: memory` — Sets the storage type for `journald` log buffers.

The `output` section:

Lists the main output options. To enable each one, set `enable: true` and specify the parameters:

-   **Elasticsearch**

```yml
  es:
    enable: false
    host: "<Elasticsearch url>"
    port: 9200
    logstashPrefix: ks-logstash-log
    bufferSize: 20MB
    traceError: true
    httpUser: "<username>"
    httpPassword: "<password>"
```

Configuration for sending logs to Elasticsearch. Specify the service URL, port, and authentication credentials.

-   **Kafka**

```yml
  kafka:
    enable: false
    brokers: "<kafka broker list>"
    topics: ks-log
    bufferSize: 20MB
    retryLimit: "10"
```

Configuration for sending logs to Kafka. Specify a comma-separated list of brokers and the topic for logs.

-   **Loki**

```yml
  loki:
    enable: false
    retryLimit: "no_limits"
    host: 127.0.0.1
    port: 3100
    httpUser: myuser
    httpPassword: mypass
    tenantId: "<tenant>"
```

Allows sending logs to Loki. Specify the host, port, credentials, and tenant ID.

-   **stdout**

```yml
  stdout:
    enable: true
```

Useful for testing, as it outputs logs to standard output (stdout).

The `filter` section:

-   **multiline**:
    -   `enable: false` — By default, multiline is disabled.
    -   You can configure parsers (`go`, `python`, `java`, etc.) or add custom ones.
-   **kubernetes**:
    -   `enable: true` — Enables the Kubernetes filter to add pod metadata, such as `namespace`, `pod`, `labels`, and `annotations`.
    -   `labels: false`, `annotations: false` — By default, these fields are not added.
-   **containerd**:
    -   `enable: true` — Enables the filter for parsing Containerd logs.
-   **systemd**:
    -   `enable: true` — Enables the filter for parsing `journald` logs.

### Fluentd

-   **enable** — If you need to use Fluentd (instead of or alongside Fluent Bit), set it to true.
-   **mode**:
    -   `collector` — Deploys as a StatefulSet.
    -   `agent` — Deploys as a DaemonSet (like Fluent Bit), meaning one agent per node.
-   **forward**:
    -   `port: 24224` — The default Fluentd input for forwarding.
-   **output** — Similar to Fluent Bit, it includes `es`, `kafka`, `opensearch`, etc.

## Removing Fluent Operator

Uninstalling the add-on via the management panel is not supported, but it can be done manually. Follow these steps:

1.  Delete the `fluent-bit` object:
    

```shell
kubectl delete -n fluent-operator fluentbit/fluent-bit
```

2.  Uninstall the `fluent-operator` Helm release:
    

```shell
helm uninstall -n fluent-operator fluent-operator
```

3.  Delete all CRDs related to `fluent-operator`:
    

```shell
kubectl delete crd $(kubectl get crd | grep fluent | awk '{print $1}')
```

4.  Check that all related resources and namespaces are removed:
    

```shell
kubectl get namespaces
```

After completing these steps, Fluent Operator will be completely removed from your cluster.