Blacklist¶

By setting up a blacklist, you can filter out different types of data that meet specific conditions. Once a blacklist is configured, data matching the conditions will no longer be reported to the Guance workspace, helping you save on data storage costs.

Prerequisites¶

• Install DataKit;

• To configure data types other than logs, DataKit version must be higher than 1.4.7;

• If a Filter is configured in the datakit.conf file, the blacklist configured here will not take effect.

Create a Blacklist¶

1. Click Manage > Blacklist > Create;

2. Define the name and description of the current blacklist rule;

3. Select the data source type;

4. Add one or more filtering rules as needed;

5. Click Confirm to enable the data blacklist filtering rule.

Data Source¶

The blacklist name is automatically generated based on the data source, including Log, Basic Objects, Resource Catalog, Network, APM, RUM, Event, Metrics, Profile.

After entering the field name, field value, and other information, it will take effect once the data source and fields are configured via DataKit and data is reported.

Filtering¶

1. Supports two condition types:

   • Any (OR condition)

   • All (AND condition)

2. Field Name: Supports manual input, must be an exact value. The field name to match can be viewed in the Explorer's "Show Columns";

3. Field Value: Supports manual input, single value, multiple values, and regular expression syntax;

4. Operator: Supports 4 modes: in / not in / match / not match. in / not in are exact matches, match / not match are regex matches:

   Operator	Supported Types	Description	Example
   in / not in	Numeric	Whether the specified field is in the list; the list supports mixed types	1,2,"foo",3.5
   match / not match	Regex	Whether the specified field matches the regex in the list; the list only supports string types	"foo.*","bar.*"

Example¶

In the following example, a blacklist named "Conditional Filtering" is defined. It selects logs from All Sources, satisfying the conditions where status is ok or info, AND host is not hz-dataflux-saas-daily-01, AND service does not contain the string kodo. Data meeting all three of these matching rules will be filtered and not reported to the workspace.

After setting the blacklist, you can check in the Explorer whether the blacklist is effective based on the filtering conditions. Once the blacklist is created and effective, data matching the filtering conditions will no longer be reported to the workspace.

Options¶

You can manage the blacklist through the following operations:

• Filter based on different data types;

• Search and locate blacklists by entering the name in the search bar;

• Enable/Disable blacklist;

• Modify already created data filtering rules;

• Delete existing filtering rules. After deletion, data will be reported to the workspace normally;

• Click to batch export or batch delete blacklists;

• Create blacklists by importing JSON files, but ensure the file is a configuration file provided by Guance.

Precautions¶

1. If a blacklist filter is configured in the datakit.conf file during DataKit installation and configuration, the blacklist rules configured in Guance will not take effect for it;

2. DataKit pulls data every 10 seconds. The blacklist configuration does not take effect immediately; it requires at least 10 seconds to wait;

3. After the blacklist configuration is completed, it is uniformly saved in the .pull file located in the DataKit directory /usr/local/datakit/data.

Further Reading¶