The idea of configuration center is to put all kinds of configurations, parameters and switches in a centralized place for unified management and provide a set of standard interfaces.
When each service needs to obtain configuration, it will configure the interface pull of the center. When various parameters in the configuration center are updated, it can also inform each service to synchronize the latest information in real time, so that it can be dynamically updated.
Adopting "centralized configuration management" can solve the traditional problem of "too scattered configuration files". All configurations are centralized in the configuration center, which does not need to bring one for each project, thus greatly reducing the development cost.
Adopting "separation of configuration and application" can solve the problem that the traditional "configuration file can't distinguish the environment". Configuration does not follow the environment. When different environments have different requirements, it can be obtained from the configuration center, which greatly reduces the operation and maintenance deployment cost.
With the function of "real-time update", it is used to solve the problem of traditional "static configuration". When the online system needs to adjust the parameters, it only needs to be dynamically modified in the configuration center.
Datakit supports multiple configuration centers, such as etcd-v3consulrediszookeeperfile, and can work together with multiple configuration centers at the same time.
When the configuration center data changes, DataKit can automatically change the configuration, add or delete collectors, and relevant collectors are restarted as necessary.
Multiple datacenter backends can be configured at the same time, and the data configuration information of each backend is merged and injected into DataKit. Any back-end information changes will be detected by DataKit, and DataKit will automatically update the relevant configuration and restart the corresponding collector.
Because of the particularity of Kubernetes environment, the installation/configuration mode with environment variable passing is the simplest.
When installing in Kubernetes, you need to set the following environment variables to bring Confd configuration information into it:
If you need to define some DataKit configuration during the installation phase, you can add environment variables to the installation command, just append them before DK_DATAWAY. Such as:
# Windows: multiple environment variables are divided by semicolons$env:NAME1="value1";$env:Name2="value2"# Linux/Mac: multiple environment variables are divided by spacesNAME1="value1"NAME2="value2"
Configuration Center can modify the configuration of these collectors, but cannot delete or stop these collectors.
If you want to delete the default collector, you can open the datakit.conf file in the DataKit conf.d directory and delete the collector in default_enabled_inputs.
Self can neither delete, stop, nor modify the configuration.
Some collectors only need to run singletons, such as all default open collectors, netstat, etc. Some can be run in multiple instances, such as nginx, nvidia_smi... and so on.
In the collector configuration of single case operation, only the data ranked first is accepted, and the latter is automatically discarded.
Datakit configuration information is stored in the data center as a Key-Value.
The prefix of Key must be /datakit/, such as /datakit/XXX , XXX is not duplicated. It is recommended to use the corresponding collector name, such as /datakit/netstat.
The contents of Value are the full contents of the various configuration files in the conf. d subdirectory. For example: