Skip to content

Dataway


Introduction

DataWay is the data gateway of Guance. All data reported by collectors to Guance must pass through the DataWay gateway.

Dataway Installation

  • Create Dataway

On the "Data Gateway" page in the Guance management backend, click "Create Dataway". Enter the name and binding address, then click "Create".

After creation, a new Dataway will be automatically created and the installation script for Dataway will be generated.

Info

The binding address is the Dataway gateway address, which must be a complete HTTP address, such as http(s)://1.2.3.4:9528, including the protocol, host address, and port. The host address can generally be the IP address of the machine where Dataway is deployed, or it can be specified as a domain name, which needs to be resolved properly.

Note: Ensure that the collector can access this address, otherwise data collection will not be successful.

  • Install Dataway
DW_KODO=http://kodo_ip:port \
   DW_TOKEN=<tkn_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX> \
   DW_UUID=<YOUR_UUID> \
   bash -c "$(curl https://static.guance.com/dataway/install.sh)"

After installation, a dataway.yaml file will be generated in the installation directory. The content example is as follows, which can be manually modified and take effect by restarting the service.

dataway.yaml (Click to expand)

```yaml

============= DATAWAY CONFIG =============

Dataway UUID, we can get it on during create a new dataway

uuid:

It's the workspace token, most of the time, it's

system worker space's token.

token:

secret_token used under sinker mode, and to check if incomming datakit

requests are valid.

secret_token:

If internal token allowed? If ok, the data/request will direct to

the workspace with the token above

enable_internal_token: false

is empty token allowed? If ok, the data/request will direct to

the workspace with the token above

enable_empty_token: false

Is dataway cascaded? For cascaded Dataway, it's remote_host is

another Dataway and not Kodo.

cascaded: false

kodo(next dataway) related configures

remote_host: http_timeout: 30s

http_max_idle_conn_perhost: 0 # default to CPU cores http_max_conn_perhost: 0 # default no limit

insecure_skip_verify: false http_client_trace: false max_conns_perhost: 0 sni: ""

dataway API configures

bind: 0.0.0.0:9528

disable 404 page

disable_404page: false

dataway TLS file path

tls_crt: tls_key:

enable pprof

pprof_bind: localhost:6060

api_limit_rate : 100000 # 100K max_http_body_bytes : 67108864 # 64MB copy_buffer_drop_size : 262144 # 256KB, if copy buffer memory larger than this, this memory released reserved_pool_size: 4096 # reserved pool size for better GC

within_docker: false

log_level: info log: log gin_log: gin.log

cache_cfg: # cache disk path dir: "disk_cache"

# disable cache disabled: false

clean_interval: "10s"

# in MB, max single data package size in disk cache, such as HTTP body max_data_size: 100

# in MB, single disk-batch(single file) size batch_size: 128

# in MB, max disk size allowed to cache data max_disk_size: 65535

# expire duration, default 7 days expire_duration: "168h"

prometheus: listen: "localhost:9090" url: "/metrics" enable: true

sinker:

etcd:

urls:

- http://localhost:2379 # one or multiple et

Feedback

Is this page helpful? ×