Customize Dialtesting
In some cases, you may not be able to connect to SAAS's dialing task service. In this case, we can define the dialing task through the local JSON file.
Configuration¶
Configure Collector¶
Go to the conf.d/network
directory under the DataKit installation directory, copy dialtesting.conf.sample
and name it dialtesting.conf
. Examples are as follows:
[[inputs.dialtesting]]
server = "file://</path/to/your/local.json>"
# Note: Taking Linux as an example, assuming your json directory is /some/path/my.json, then the
# server should be written as file:///some/path/my.json
# Note that the following tag suggestions are filled in one by one (do not modify the tag key here), so that the complete dialing test results can be displayed on the page.
[inputs.dialtesting.tags]
country = "<specify-datakit-country>" # Countries where DataKit is deployed
province = "<specify-datakit-province>" # Provices where DataKit is deployed
city = "<specify-datakit-city>" # Cities where DataKit is deployed
isp = "<specify-datakit-ISP>" # Specify the network service provider where DataKit is located
region = "<your-region>" # You can specify a region name at will
The collector can now be turned on by ConfigMap injection collector configuration.
The specific country/region and ISP selection can be selected as shown in the following figure (note that you don't really create a new "self-built node", just provide an alternative source here):
Configure the Dial Test Task¶
At present, the dialing test task supports four dialing test types, namely HTTP, TCP, ICMP and WEBSOCKET services. The JSON format is as follows:
The following is a specific dialing test example:
{
"HTTP": [
{
"name": "baidu-json-test",
"method": "GET",
"url": "http://baidu.com",
"post_url": "https://<your-dataway-host>?token=<your-token>",
"status": "OK",
"frequency": "10s",
"success_when_logic": "and",
"success_when": [
{
"response_time": "1000ms",
"header": {
"Content-Type": [
{
"contains": "html"
}
]
},
"status_code": [
{
"is": "200"
}
]
}
],
"advance_options": {
"request_options": {
"auth": {}
},
"request_body": {}
},
"update_time": 1645065786362746
}
],
"TCP": [
{
"name": "tcp-test",
"host": "www.baidu.com",
"port": "80",
"status": "OK",
"frequency": "10s",
"success_when_logic": "or",
"success_when": [
{
"response_time": [
{
"is_contain_dns": true,
"target": "10ms"
}
]
}
],
"update_time": 1641440314550918
}
]
}
After editing this JSON, it is recommended to find some(online tools or this tool)to verify that the JSON format is correct. If the JSON format is incorrect, the dialing test will not take effect.
After configuration, restart DataKit.
Test Task Field Definition¶
The dialing task fields include "public fields" and "additional fields" for specific dialing tasks.
Public Field¶
The public fields of dialing test tasks are defined as follows:
Field | Type | Whether Required | Description |
---|---|---|---|
name |
string | Y | Dial test service name |
status |
string | Y | Dial test service status, such as "OK"/"stop" |
frequency |
string | Y | Dial frequency |
success_when_logic |
string | N | The logical relationship between success_when conditions, such as "and"/"or", defaults to "and" |
success_when |
object | Y | See below for details |
advance_options |
object | N | See below for details |
post_url |
string | N | Send the dialing test result to the workspace pointed by the Token, and if it is not filled in, send it to the workspace where the current DataKit is located |
tags_info |
string | N | Custom tags, such as t1,t2 |
workspace_language |
string | N | Workspace language, such as zh ,en |
HTTP Dial Test¶
Extra field:
Field | Type | Whether Required | Description |
---|---|---|---|
method |
string | Y | HTTP request method |
url |
string | Y | Complete HTTP request address |
The overall JSON structure is as follows:
{
"HTTP": [
{
"name": "baidu-json-test",
"method": "GET",
"url": "http://baidu.com",
"post_url": "https://<your-dataway-host>?token=<your-token>",
"status": "OK",
"frequency": "10s",
"success_when_logic": "and",
"success_when": ...,
"advance_options": ...
},
{
... another HTTP dialtesting
}
]
}
success_when
Definition¶
The judging conditions used to define the success of dialing test mainly include the following aspects:
- HTTP request returns body judgment(
body
)
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | Whether the returned body is equal to the specified field |
is_not |
string | N | Whether the returned body is not equal to the specified field |
match_regex |
string | N | Whether the returned body contains a substring of the matching regular expression |
not_match_regex |
string | N | Whether the returned body does not contain a substring of the matching regular expression |
contains |
string | N | Whether the returned body contains the specified substring |
not_contains |
string | N | Whether the returned body does not contain the specified substring |
eg.
Here, body
can configure multiple verification rules, and the relationship between them is determined by "success_when_logic". When it is configured as and
, if any rule is verified, it will be considered that the current dialing test failed; When it is configured to or
, if any rule is verified, it will be considered that the current dialing test is successful. The default is an and
relationship. The following verification rules all follow this judgment principle.
Note that the regular is escaped correctly here, and the actual regular expression in the example is
\d\d.*
.
- HTTP request returns header judgment (
header
)
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | The header returned specifies whether the field is equal to the specified value |
is_not |
string | N | The header returned specifies whether the field is not equal to the specified value. |
match_regex |
string | N | The header returned specifies whether the field contains a substring of the matching regular expression. |
not_match_regex |
string | N | The header returned specifies whether the field does not contain a substring of the matching regular expression. |
contains |
string | N | The header returned specifies whether the field contains the specified substring. |
not_contains |
string | N | The header returned specifies whether the field does not contain the specified substring. |
for example:
Because there may be decisions for multiple types of headers, validation for multiple headers can also be configured here:
"success_when": [
{
"header": {
"Content-Type": [
{
"contains": "some-header-value"
}
],
"Etag": [
{
"match_regex": "\\d\\d-.*"
}
]
}
}
]
- HTTP request returns status code (
status_code
)
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | Whether the status code returned is equal to the specified field |
is_not |
string | N | Whether the status code returned is not equal to the specified field |
match_regex |
string | N | Whether the status code returned contains a substring of the matching regular expression |
not_match_regex |
string | N | Whether the status code returned does not contain a substring of the matching regular expression |
contains |
string | N | Whether the status code returned contains the specified substring |
not_contains |
string | N | Whether the status code returned does not contain the specified substring |
for example:
For a certain URL dial test, its HTTP return is usually only one, so only one validation rule is generally configured here (although multiple array configurations are supported).
- HTTP request response time (
response_time
)
Only one time value can be filled in here. If the response time of the request is less than the specified value, the dialing test is judged to be successful, such as:
Note that the time units specified here are
ns
(nanoseconds)/us
(microseconds) /ms
(milliseconds) /s
(seconds) /m
(minutes) /h
(hours). For HTTP dial testing,ms
units are generally used.
Several kinds of judgment basis listed above can be used in combination, and the relationship between them is determined by "success_when_logic". When it is configured as and
, if any rule is verified, it is considered that the current dialing test fails; When it is configured to or
, if any rule is verified, it will be considered that the current dialing test is successful; The default is an and
relationship. Such as:
"success_when": [
{
"response_time": "1000ms",
"header": { HTTP header 相关判定 },
"status_code": [ HTTP 状态码相关判定 ],
"body": [ HTTP body 相关判定 ]
}
]
advance_options
Definition¶
Advanced options are mainly used to adjust specific dialing behavior, mainly in the following aspects:
- HTTP Request Option (
request_options
)
Field | Type | Whether Required | Description |
---|---|---|---|
follow_redirect |
bool | N | Whether redirect jump is supported |
headers |
map[string]string | N | Specify a set of headers on an HTTP request |
cookies |
string | N | Specify the requested Cookie |
auth |
object | N | Specify the authentication method of the request |
Among them, auth
only supports ordinary username and password authentication, which is defined as follows:
Field | Type | Whether Required | Description |
---|---|---|---|
username |
string | Y | User name |
password |
string | Y | User name and password |
request_options
example:
"advance_options": {
"request_options": {
"auth": {
"username": "zhangsan",
"password": "fawaikuangtu"
},
"headers": {
"X-Prison-Breaker": "zhangsan",
"X-Prison-Break-Password": "fawaikuangtu"
},
"follow_redirect": false
},
}
- HTTP Request Body(
request_body
)
Field | Type | Whether Required | Description |
---|---|---|---|
body_type |
string | N | Body type, that is, the value of the request header Content-Type |
body |
string | N | Request Body |
request_body
example:
"advance_options": {
"request_body": {
"body_type": "text/html",
"body": "Fill in the request body, and pay attention to various complicated escapes here"
}
}
- HTTP Request a Certificate (
certificate
)
Field | Type | Whether Required | Description |
---|---|---|---|
ignore_server_certificate_error |
bool | N | Whether to ignore certificate errors |
private_key |
string | N | key |
certificate |
string | N | Certificate |
ca |
string | N | Temporarily unused |
certificate
example:
"advance_options": {
"certificate": {
"ignore_server_certificate_error": false,
"private_key": "<your-private-key>",
"certificate": "<your-certificate-key>"
},
}
private_key
example:
-----BEGIN PRIVATE KEY-----
MIIxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxNn+/x
9WKHZvRf3lbLY7GAR/emacU=
-----END PRIVATE KEY-----
Here is an example of certificate
:
-----BEGIN CERTIFICATE-----
MIIxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxWDR/+
InEHyg==
-----END CERTIFICATE-----
Under Linux, this pair of keys can be generated by the following command:
- HTTP Request broker (
proxy
)
Field | Type | Whether Required | Description |
---|---|---|---|
url |
string | N | 代理的 The URL of the proxy, such as http://1.2.3.4:4321 |
headers |
map[string]string | N | Specify a set of headers on an HTTP request |
proxy
example:
"advance_options": {
"request_options": {
"proxy": {
"url": "http://1.2.3.4:4321",
"headers": {
"X-proxy-header": "my-proxy-foobar"
}
}
},
}
TCP Dial Test¶
Extra Field¶
Field | Type | Whether Required | Description |
---|---|---|---|
host |
string | Y | TCP Host address |
port |
string | Y | TCP Port |
timeout |
string | N | TCP connection timeout |
message |
string | N | TCP message sent |
The complete JSON structure is as follows:
{
"TCP": [
{
"name": "tcp-test",
"host": "www.baidu.com",
"port": "80",
"message": "hello",
"timeout": "10ms",
"enable_traceroute": true,
"post_url": "https://<your-dataway-host>?token=<your-token>",
"status": "OK",
"frequency": "60s",
"success_when_logic": "and",
"success_when": [
{
"response_time":[
{
"is_contain_dns": true,
"target": "10ms"
}
],
"response_message": [
{
"is": "hello"
}
],
"hops": [
{
"op": "eq",
"target": 20
}
]
}
]
}
]
}
success_when
Definition¶
- TCP Response Time Determination (
response_time
)
response_time
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
target |
string | Y | Determining whether the response time is less than the value |
is_contain_dns |
bool | N | Indicates whether the response time includes DNS resolution time |
- Return a message decision (
response_message
)
response_message
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | Whether the returned message is equal to the specified field |
is_not |
string | N | Whether the returned message is not equal to the specified field |
match_regex |
string | N | Whether the returned message contains a substring of the matching regular expression |
not_match_regex |
string | N | Whether the returned message does not contain a substring of the matching regular expression |
contains |
string | N | Whether the returned message contains the specified substring |
not_contains |
string | N | Whether the returned message does not contain the specified substring |
for example:
- Network hop count (
hops
)
hops
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
op |
string | Y | Compare relation, retrievable eq(=),lt(<),leq(<=),gt(>),geq(>=) |
target |
float | Y | Decision value |
ICMP Dial Test¶
Extra Field¶
Field | Type | Whether Required | Description |
---|---|---|---|
host |
string | Y | Host address |
packet_count |
int | N | Number of ICMP packets sent |
timeout |
string | N | Connection timeout |
The complete JSON structure is as follows:
{
"ICMP": [
{
"name": "icmp-test",
"host": "www.baidu.com",
"timeout": "10ms",
"packet_count": 3,
"enable_traceroute": true,
"post_url": "https://<your-dataway-host>?token=<your-token>",
"status": "OK",
"frequency": "10s",
"success_when_logic": "and",
"success_when": [
{
"response_time": [
{
"func": "avg",
"op": "leq",
"target": "50ms"
}
],
"packet_loss_percent": [
{
"op": "leq",
"target": 20
}
],
"hops": [
{
"op": "eq",
"target": 20
}
],
"packets": [
{
"op": "geq",
"target": 1
}
]
}
]
}
]
}
success_when
Definition¶
- ICMP packet loss rate (
packet_loss_percent
)
Fill in the specific value as an array object, and each object parameter is as follows:
Field | Type | Whether Required | Description |
---|---|---|---|
op |
string | Y | Compare relationship retrievable eq(=),lt(<),leq(<=),gt(>),geq(>=) |
target |
float | Y | Decision value |
- ICMP response time (
response_time
)
Fill in the specific time as an array object, and each object parameter is as follows:
Field | Type | Whether Required | Description |
---|---|---|---|
func |
string | Y | Statistical type, take the value avg,min,max,std |
op |
string | Y | Comparison relation, take value eq(=),lt(<),leq(<=),gt(>),geq(>=) |
target |
string | Y | Decision value |
- Network hop count (
hops
)
hops
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
op |
string | Y | Compare relationships, retrievable eq(=),lt(<),leq(<=),gt(>),geq(>=) |
target |
float | Y | Decision value |
- Number of packets grabbed (
packets
)
packets
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
op |
string | Y | Compare relationship retrievable eq(=),lt(<),leq(<=),gt(>),geq(>=) |
target |
float | Y | Decision value |
WEBSOCKET Dial Test¶
Extra Field¶
Field | Type | Whether Required | Description |
---|---|---|---|
url |
string | Y | Websocket connection address, such as ws://localhost:8080 |
message |
string | Y | Websocket message sent after successful connection |
The complete JSON structure is as follows:
{
"WEBSOCKET": [
{
"name": "websocket-test",
"url": "ws://localhost:8080",
"message": "hello",
"post_url": "https://<your-dataway-host>?token=<your-token>",
"status": "OK",
"frequency": "10s",
"success_when_logic": "and",
"success_when": [
{
"response_time": [
{
"is_contain_dns": true,
"target": "10ms"
}
],
"response_message": [
{
"is": "hello1"
}
],
"header": {
"status": [
{
"is": "ok"
}
]
}
}
],
"advance_options": {
"request_options": {
"timeout": "10s",
"headers": {
"x-token": "aaaaaaa",
"x-header": "111111"
}
},
"auth": {
"username": "admin",
"password": "123456"
}
}
}
]
}
success_when
Definition¶
- Response time judgment (
response_time
)
response_time
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
target |
string | Y | Determining whether the response time is less than the value |
is_contain_dns |
bool | N | Indicates whether the response time includes DNS resolution time |
- Return a message decision (
response_message
)
response_message
is an array object with the following parameters for each object:
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | Whether the returned message is equal to the specified field |
is_not |
string | N | Whether the returned message is not equal to the specified field |
match_regex |
string | N | Whether the returned message contains a substring of the matching regular expression |
not_match_regex |
string | N | Whether the returned message does not contain a substring of the matching regular expression |
contains |
string | N | Whether the returned message contains the specified substring |
not_contains |
string | N | Whether the returned message does not contain the specified substring |
for example:
- Request to return header judgment(
header
)
header
is a dictionary type object whose value for each object element is an array object with the following parameters:
Field | Type | Whether Required | Description |
---|---|---|---|
is |
string | N | The header returned specifies whether the field is equal to the specified value |
is_not |
string | N | The header returned specifies whether the field is not equal to the specified value |
match_regex |
string | N | The header returned specifies whether the field contains a substring of the matching regular expression |
not_match_regex |
string | N | The header returned specifies whether the field does not contain a substring of the matching regular expression |
contains |
string | N | The header returned specifies whether the field contains the specified substring |
not_contains |
string | N | The header returned specifies whether the field does not contain the specified substring |
for example:
advance_options
Definition¶
- Request option (
request_options
)
Field | Type | Whether Required | Description |
---|---|---|---|
timeout |
string | N | Connection timeout |
headers |
map[string]string | N | Specify a set of headers on request |
"advance_options": {
"request_options": {
"timeout": "30ms",
"headers": {
"X-Token": "xxxxxxxxxx"
}
},
}
- Authentication information (
auth
)
Support for common user name and password authentication (Basic access authentication)。
Field | Type | Whether Required | Description |
---|---|---|---|
username |
string | Y | user name |
password |
string | Y | user name and password |