Skip to content

Blacklist

By setting up a blacklist, you can filter out different types of data that meet specific conditions. After configuring the blacklist, data matching the conditions will no longer be reported to the Guance workspace, helping you save on data storage costs.

Prerequisites

  1. Install DataKit.

  2. To configure data types other than logs, the DataKit version must be higher than 1.4.7.

  3. If a Filter is configured in the datakit.conf file, the blacklist configured here will no longer take effect.

Create a Blacklist

  1. Click Manage > Blacklist > Create.

  2. Define the name and description for the current blacklist rule.

  3. Select the Data Source Type.

  4. Add one or more Filter Rules as needed.

  5. Click Confirm to enable the data blacklist filtering rule.

Note

  • If you only need to create a blacklist for log data, you can go directly to Log > Blacklist for configuration.

  • Data types support string, integer, and float.

  • If the data source is logs, a log filtering rule will be synchronously created under the Log > Blacklist menu, and vice versa.

Data Sources

The blacklist name is automatically generated based on the data source, including Log, Basic Objects, Resource Catalog, Network, APM, RUM, Event, Metrics, Profile.

After entering the field name, field value, and other information, it will take effect once the data source and fields are configured and data is reported via DataKit.

Data Type Data Source (Supports custom presets)
Log Log source (source), e.g., nginx
Basic Objects Class (class), e.g., HOST
Resource Catalog Class (class), e.g., MySQL
Network Source (source), e.g., netflow, httpflow
APM Service (service), e.g., redis; you can select "All Services"
RUM Application (app_id)
Event Source (source), e.g., monitor
Metrics Measurement, e.g., cpu
Profile Service (service)

Filter Rules

Configuration Item Description
Condition Relation
  • Any (OR)：Filter if any rule is satisfied.
  • All (AND)：Filter only if all rules are satisfied.
    		•
    Field Name The field name from the Explorer's Display Columns. Case-sensitive, requires exact match.
    Field Value Supports single value, multiple values (separated by English commas), and regular expressions.
    Operator
  • in (Exact match, contains)
  • not in (Exact match, does not contain)
  • match (Regular expression match)
  • not match (Regular expression does not match)
    		•

    Field Value Format Description

    Type Input Format Example
    Single Value Direct input error
    Multiple Values Separated by English commas, no spaces ok,info,debug
    Regular Expression Direct input of regular expression ^test-.*

    Operator Description

    1. Regular Expression Syntax

    Uses Go standard regular expression syntax (RE2).

    Common Examples:

    • ^prod-：Matches strings starting with prod-.

    • .*\.test\.：Matches strings containing .test..

    2. Exact Match Description

    in / not in are exact matches and are case-sensitive. For example, status in error only matches error, not ERROR or error_msg.

    Rule Example

    In the following example:

    1. Define the blacklist name as "Network Devices".
    2. No description is added.
    3. Select to filter Basic Objects data, and only filter object data collected via SNMP.

    4. Select the "All" condition relation and define a rule: SNMP network device data with the device name switch-s5700 will not be reported to Guance.

      • Field Name：device_hostname
      • Operator：in (contains)
      • Field Value：switch-s5700

    5. Click Confirm.

    The above example is a standard blacklist usage scenario:

    A user collects network device data via SNMP but does not want to monitor a specific device (switch-s5700). They use the blacklist to filter out data from that device, saving storage costs.

    Verify Blacklist Effectiveness

    1. Wait for it to take effect: After configuration, wait at least 10 seconds (DataKit pulls configuration every 10 seconds).
    2. Compare data: In the Explorer, select the same time period before and after configuration to compare if the number of data entries has decreased.
    3. Check configuration distribution: On the DataKit host, check the /usr/local/datakit/data/.pull file to confirm the blacklist configuration has been saved.

    List Operations

    You can manage the blacklist list through the following operations:

    • Filter based on different data types.

    • Use the search bar to search and locate by blacklist name.

    • Enable/Disable a blacklist.

    • Modify already created data filtering rules.

    • Delete existing filtering rules. After deletion, data will be reported to the workspace normally.

    • Click to batch export or batch delete blacklists.

    • Create a blacklist by importing a JSON file, but ensure the file is a configuration file provided by Guance.

    Notes

    1. If a blacklist filter is configured in the datakit.conf file during DataKit installation and configuration, the blacklist rules configured in Guance will not take effect for it.

    2. DataKit pulls data every 10 seconds. Therefore, the blacklist configuration does not take effect immediately and requires waiting at least 10 seconds.

    3. After the blacklist configuration is completed, it is uniformly saved in the .pull file under the DataKit directory /usr/local/datakit/data.

    4. Blacklist filtering is executed on the DataKit side. Filtered data will not be transmitted to Guance and cannot be recovered.

    Further Reading

    Feedback

    Is this page helpful? ×