Blacklist¶

Guance supports filtering out different types of qualified data by setting blacklist, that is, after configuring blacklist, qualified data will no longer be reported to Guance workspace, which helps you save data storage costs.

Preconditions¶

Install DataKit
If you need to configure other data besides logs, the DataKit version must be higher than 1.4.7.

Create¶

In Guance workspace, click Manage > Blacklist > Create.

In Data Source, select the data type, add one or more filtering rules in Filter and click Confirm to open the data blacklist filtering rules.

Data source: Blacklist names are automatically generated according to data sources, Support the selection of Logs, Basic Object, Custom Object, Network, APM, RUM, Security Check, Event, Metrics and Profile, and support the manual input of preset blacklist, package data source and field name, which will take effect after configuring data source and field through DataKit and reporting data.

Data Type	Data source (support custom preset)
Log	log source, such as nginx, etc.
Basic object	class, such as HOOT, etc.
Custom object	class, such as MySQL, etc
Network	source, such as netflow and httpflow
APM	services, such as redis, support the selection of all service
RUM	apply (app_id)
Scheck	category, such as system, etc.
Events	source, such as monitor, etc
Metrics	measurement, such as cpu, etc.
Profile	service

Filtering: Support two conditional choices, "any" and "all". "Any" is an OR (OR) condition AND "All" is an AND (AND "condition.
Field name: Manual input of field name is supported, which must be an accurate value. You can view the field name to be matched in the "Display Column" of the explorer.
Field value: Support manual input of field value, input of single value, multi-value and regular syntax.
Operator: Four modes of in / not in / match / not match are supported, in / not in is precise match, and match / not match is regular match.

Operator	Available numeric types	Description	Example
`in / not in`	Numeric List	Whether the specified field is in a list, and multi-type cluttering is supported in the list.	`1,2,"foo",3.5`
`match / not match`	Regular expression list	Whether the specified field matches the regular in the list, which only supports string types.	`"foo.","bar."`

Note: Data types support string, integer and floating point types.

If the selected data source is "log", create a log filtering rule synchronously under the function menu Log > Blacklist, and vice versa.

Example¶

In the following example, a new blacklist is created, a log that selects All Sources, satisfies status as ok or info, and host is not hz-dataflux-saas-daily-01, and the service does not contain the word kodo, meaning that data that meets all three matching rules will be filtered and no longer reported to the workspace.

After setting the blacklist, you can check whether the blacklist is effective according to the filter criteria in the explorer. After the blacklist is created and takes effect, the data that meets the filtering conditions will no longer be reported to the workspace.

Edit¶

On the right side of the blacklist, click Edit to edit the created data filtering rules. In the following example, when the blacklist is set, the log for All Sources satisfies status as ok or info, or host as hz-dataflux-saas-daily-01, or service does not contain the word kodo, i.e. data satisfying any of these three matching rules is filtered and no longer reported to the workspace.

Delete¶

On the right side of the blacklist, click Delete to delete the existing filtering rules. After the filtering rules are deleted, the data will be reported to the workspace normally.

Batch¶

In the Guance workspace Manage > Blacklist, click Batch to Batch Export or Batch Delete blacklist.

Note: This function is only displayed for workspace Owner, Administrators and Standard members.

Import/Export¶

Support Import/Export Blacklist in Manage > Blacklist of Guance workspace, that is, create blacklist by importing/exporting JSON files.

Note: The imported JSON file needs to be the configuration JSON file from Guance.

Notes¶

Blacklist rules configured in Guance will not take effect if the blacklist is configured in the file datakit.conf under the directory /usr/local/datakit/conf.d when installing and configuring the datakit;
DataKit pulls data every 10 seconds, and the blacklist will not take effect immediately after configuration, so it needs to wait at least 10 seconds.
After the blacklist is configured, it is stored in the .pull file under the datakit directory /usr/local/datakit/data, and more can be found in the documentation view blacklist.