Skip to content

Data Forwarding

For data with long storage cycles and low update frequencies, the data forwarding feature can be used to save data, including logs, to object storage or forward it to an external storage system.

After the rule takes effect, on the data forwarding page, you can quickly search for stored data by setting query times and data forwarding rules.

Prerequisites

Commercial Plan only.

Create

Navigate to the Data Forwarding > Forwarding Rules > Create page.

Once the data forwarding rule is created, the system will perform a rule validation every 5 minutes.

Input Rule Name

This is the name of the current data forwarding rule.

  • Include extended fields: By default, only the message field content of logs that meet the conditions will be forwarded. If "Include extended fields" is checked, the entire log data that meets the conditions will be forwarded. Application performance and user access data are forwarded as full records by default and are not affected by this option.
Note

When creating multiple data forwarding rules, priority is given to matching rules that include extended fields. If different rules match the same data, the rule with extended fields will display the entire log data.

Define Filtering Conditions

  1. Data source: Includes logs, application performance, user access, events, audit events.

  2. Filtering conditions: Supports custom logic between conditions; multiple conditions can be added.

    • All conditions: Only logs that meet all filtering conditions will be saved in the data forwarding;

    • Any condition: Logs meeting any one of the filtering conditions will be saved in the data forwarding.

Condition operators are shown in the table below:

Condition Operator Match Type
in, not in Exact match, supports multiple values (comma-separated)
match, not match Fuzzy match, supports regular expression syntax

If no filtering conditions are added here, it means all data will be saved.

Select Archiving Type

To provide more comprehensive data forwarding storage methods, the system supports five storage paths.

Guance: When choosing Guance as the data forwarding storage object, matched log data will be saved in Guance's OSS, S3, OBS object storage.

AWS S3;

Huawei Cloud OBS;

Alibaba Cloud OSS;

Kafka MESSAGE QUEUES.

Note

  • All five archiving types are available across all sites;

  • When choosing Guance as the data forwarding storage object, the minimum log storage period is 180 days by default, and once the rule is created, it cannot be canceled; daily charges will apply during the storage period. You can modify this under Manage > Settings > Change Data Storage Policy.

Encrypted Storage

For AWS S3, Huawei Cloud OBS, Alibaba Cloud OSS data archiving types, you can enable encrypted storage to ensure the security of data forwarded from the Guance platform to the specified external storage of the enterprise.

Once enabled, the system will encrypt the forwarded data using symmetric encryption. If you need to query or view this data later, the system can decrypt and display the original content.

What is symmetric encryption?

Symmetric encryption is an encryption method that uses the same key for both encryption and decryption, like a single key that can both lock and unlock the same lock.

For more details, refer to Symmetric Encryption.

Define Data Viewing Permissions

Set viewing permissions for the forwarded data to enhance data security.

All members of the workspace can view the forwarded data.

Specify roles that can view the forwarded data.

Manage Forwarding Rules

All created data forwarding rules can be viewed in the forwarding rules list. You can manage the list through the following operations:

  • Search by entering the rule name;

  • Enable or disable the current rule;

  • Click the search, edit, or delete button on the right side of the rule to perform corresponding actions;

  • Optionally select multiple rules for batch operations.

Note
  • Viewing forwarded data may have up to a 1-hour delay;
  • In editing mode, access type and region cannot be adjusted; for rules selecting Guance storage, editing and viewing contents remain consistent;
  • Deleting a rule will not delete already forwarded data, but no new data will be generated.

Data Viewing

On the "Data Viewing" page, you can search for the latest data results based on time range and forwarding rules. You can also directly search fields in the format key:value.

The system retrieves file search matches in batches according to the selected time, returning 50 entries per batch. If the initial query does not return data or returns fewer than 50 entries, you can manually click "Continue Query" until the scan is complete.

Search Query Logic

You can use the field :~ value format for search queries, which uses regular expression matching logic.

  • The system will return all content containing the field : value string, meaning if you input a search condition like host:~ hangzhou, all data containing host: hangzhou will be returned, but exact matching is not supported;
  • The relationship between multiple filtering conditions is OR.

Data Forwarding Query Duration

After configuring the query duration, when viewing forwarded data, the query time interval will be influenced by the duration configuration set here.

  1. Navigate to Manage > Workspace Settings > Advanced Settings > Configure Query Duration;
  2. Select duration;
  3. Confirm.

Feedback

Is this page helpful? ×