Blacklist¶

By setting up a blacklist, you can filter out different types of data that meet specific conditions. Once a blacklist is configured, data matching the conditions will no longer be reported to the Guance workspace, helping you save on data storage costs.

Prerequisites¶

• Install DataKit;

• To configure data types other than logs, the DataKit version must be higher than 1.4.7;

• If a Filter is configured in the datakit.conf file, the blacklist configured here will not take effect.

Create¶

1. Click Manage > Blacklist > Create;

2. Define the name and description for the current blacklist rule;

3. Select the data source type;

4. Add one or more filtering rules as needed;

5. Click OK to enable the data blacklist filtering rule.

Data Source¶

The blacklist name is automatically generated based on the data source, including logs, basic objects, Resource Catalog, network, APM, RUM, events, Metrics, Profile.

After entering the field name, field value, and other information, it will take effect once the data source and fields are configured and data is reported via DataKit.

Filtering¶

1. Supports two condition selections:

   • Any (OR condition)

   • All (AND condition)

2. Field Name: Supports manual input, must be an exact value. The field name to be matched can be viewed in the Explorer's "Display Columns";

3. Field Value: Supports manual input, supports single value, multiple values, and regular expression syntax;

4. Operator: Supports 4 modes: in / not in / match / not match. in / not in are exact matches, match / not match are regex matches:

   Operator	Supported Types	Description	Example
   in / not in	Numeric	Whether the specified field is in the list. The list supports mixed types	1,2,"foo",3.5
   match / not match	Regex	Whether the specified field matches the regex in the list. The list only supports string types	"foo.*","bar.*"

Example¶

In the following example, a blacklist named "Conditional Filtering" is defined. Logs from All Sources are selected that satisfy status being ok or info, AND host is not hz-dataflux-saas-daily-01, AND service does not contain the word kodo. Data meeting all three matching rules simultaneously will be filtered and not reported to the workspace.

After setting the blacklist, you can check in the Explorer whether the blacklist is effective based on the filtering conditions. Once the blacklist is created and effective, data matching the filtering conditions will no longer be reported to the workspace.

Options¶

You can manage the blacklist through the following operations:

• Filter based on different data types;

• Search and locate by entering the blacklist name in the search bar;

• Enable/Disable blacklist;

• Modify already created data filtering rules;

• Delete existing filtering rules. After deletion, data will be reported to the workspace normally;

• Click to batch export or batch delete blacklists;

• Create blacklists by importing JSON files, but ensure the file is a configuration file provided by Guance.

Notes¶

1. If a blacklist filter is configured in the datakit.conf file during DataKit installation and configuration, the blacklist rules configured in Guance will not take effect for it;

2. DataKit pulls data every 10 seconds. The blacklist configuration does not take effect immediately; it requires waiting at least 10 seconds;

3. After the blacklist configuration is completed, it is uniformly saved in the .pull file located in the DataKit directory /usr/local/datakit/data.

More Reading¶