DataKit Operator¶

DataKit Operator is a project integrated with DataKit in Kubernetes orchestration, aimed at facilitating easier deployment of DataKit, as well as providing other functions such as verification and injection.

Overview¶

DataKit Operator provides automated injection capabilities for Kubernetes clusters through the Kubernetes Admission Controller mechanism, helping users integrate observability more easily. Key features include:

• DDTrace Injection: Automatically inject APM tracing agents for Java applications
• Log Collection: Automatically collect container logs via logfwd sidecar
• Performance Profiling: Inject Flameshot or Profiler components for application performance monitoring
• Configuration Management: Support both global configuration and declarative configuration injection methods

Core Advantages:

• Automated Deployment: No need to manually modify application YAML, reducing configuration errors
• Batch Management: Implement batch injection through namespace and label selectors
• Flexible Configuration: Support JSON configuration and fine-grained control via Annotations
• Version Compatibility: Maintain backward compatibility and support smooth upgrades

Prerequisites¶

• Kubernetes v1.24.1 or higher is recommended, with internet access (to download yaml files and pull corresponding images)
• Ensure MutatingAdmissionWebhook and ValidatingAdmissionWebhook controllers are enabled
• Ensure admissionregistration.k8s.io/v1 API is enabled

Installation¶

$ kubectl create namespace datakit
$ wget https://static.guance.com/datakit-operator/datakit-operator.yaml
$ kubectl apply -f datakit-operator.yaml
$ kubectl get pod -n datakit

NAME                               READY   STATUS    RESTARTS   AGE
datakit-operator-f948897fb-5w5nm   1/1     Running   0          15s

$ helm install datakit-operator datakit-operator \
    --repo https://pubrepo.guance.com/chartrepo/datakit-operator \
    -n datakit --create-namespace

$ helm -n datakit list

$ helm -n datakit get values datakit-operator -a -o yaml > values.yaml
$ helm upgrade datakit-operator datakit-operator \
    --repo https://pubrepo.guance.com/chartrepo/datakit-operator \
    -n datakit \
    -f values.yaml

$ helm uninstall datakit-operator -n datakit

Configuration Explanation¶

DataKit Operator configuration is in JSON format, stored separately as a ConfigMap in Kubernetes, and loaded into the container as environment variables.

{
    "server_listen": "0.0.0.0:9543", // Operator service listening address
    "log_level": "info",             // Operator log level
    "admission_inject_v2": {         // Injection configuration v2
        "ddtraces": [...],           // DDTrace configuration array
        "logfwds": [...],            // Log forwarding configuration array
        "flameshots": [...]          // Profiling configuration array
    },
    "admission_mutate": {            // Configuration mutation
        "loggings": [...]            // Log configuration mutation
    }
}

{
    "server_listen": "0.0.0.0:9543",
    "log_level":     "info",
    "admission_inject": {
        "ddtrace": {...},
        "profiler": {...},
        "logfwd": {...}
    },
    "admission_mutate": {
        "loggings": [...]
    }
}

Injection Methods¶

DataKit Operator supports two resource input methods:

1. Selector Configuration Injection (Imperative)

   Specify the Namespace and Selector of the target Pod by modifying the DataKit-Operator config. If a Pod meets the conditions, injection is performed.

   Advantages: No need to add Annotations to the target Pod (but the target Pod needs to be restarted)

   Disadvantages: Scope is not precise enough, potentially leading to invalid injections

2. Annotation Configuration Injection (Declarative)

   Add Annotations to the target Pod to enable its own injection.

   Advantages: Injection rejection can be precisely controlled via Annotation

   Disadvantages: Injection cannot be triggered solely by Annotation; matching rules still need to be configured, meaning besides enabling injection in the target Pod annotation, other fields in the Operator configuration are also required.

Selector Configuration Injection¶

Batch injection can be achieved by configuring namespace_selectors and label_selectors.

In the admission_inject_v2 configuration, namespace_selectors and label_selectors are configured directly in the array item. Taking DDTrace injection as an example:

{
    "admission_inject_v2": {
        "ddtraces": [
            {
                "namespace_selectors": ["testns"],
                "label_selectors":     ["app=log-output"],
                ...
            }
        ]
    }
}

• namespace_selectors: Namespace selector array, supports regular expression matching. For exact matching, surround the pattern with ^ and $, e.g., ^testns$
• label_selectors: Label selector array, uses Kubernetes Label Selector syntax

If both selectors are configured, the target Pod must satisfy both conditions simultaneously. For guidelines on writing label selectors, refer to this official documentation.

Annotation Configuration Injection¶

Adding specific Annotations to Deployment can control whether injection is allowed. Note that Annotations should be added in the template.

Supported Annotations are as follows:

Example:

    annotations:
    admission.datakit/ddtrace.enabled: "true"
    admission.datakit/logfwd.enabled: "true"

Injection Method Summary

• Global Configuration: Suitable for batch scenarios, controlling injection scope via Operator configuration
• Annotation Configuration: Suitable for fine-grained control, deciding whether to inject via Pod annotations
• Priority: Annotation configuration takes precedence over global configuration, useful for rejecting injection
• Compatibility: Supported features vary slightly across versions, please refer to specific version notes

Supported Injection Function List¶

Downward API¶

In DataKit Operator v1.4.2 and later versions, envs supports Kubernetes Downward API environment variable value fields. The following are currently supported:

For example, if there is a Pod named nginx-123 in the middleware namespace, and you want to inject the environment variables POD_NAME and POD_NAMESPACE, refer to the following:

{
    "admission_inject": {
        "ddtrace": {
            "envs": {
                "POD_NAME":      "{fieldRef:metadata.name}",
                "POD_NAMESPACE": "{fieldRef:metadata.namespace}"
            }
        }
    }
}

Ultimately, in that Pod you can see:

$ env | grep POD
POD_NAME=nginx-123
POD_NAMESPACE=middleware

FAQ¶

How to disable injection for a specific Pod?¶

Add Annotation "admission.datakit/enabled": "false" to that Pod, and no operations will be performed for it. This has the highest priority.

How does it work?¶

DataKit-Operator uses Kubernetes Admission Controller function for resource injection. For detailed mechanisms, please check the official documentation

What to note in AWS EKS environment?¶

Deploying in an AWS EKS environment may cause DataKit-Operator not to take effect; you need to open port 9543 in the security group.

Troubleshooting Guide¶