Create Forwarding Rules¶
Prerequisites¶
Only available for Commercial Plan.
Create¶
Go to the Data Forwarding > Forwarding Rules > Create page.
After the data forwarding rule is created, the system will perform rule validation every 5 minutes.
1. Enter Rule Name¶
This is the name of the current data forwarding rule.
- Include Extended Fields: By default, only the
messagefield of the log that meets the conditions is forwarded. If "Include Extended Fields" is checked, the entire log data that meets the conditions will be forwarded. APM and RUM data are forwarded in their entirety by default and are not affected by this option.
Note
When creating multiple data forwarding rules, rules that include extended fields are matched first. If different rules match the same piece of data, the logic of including extended fields will take precedence to display the entire log data.
2. Define Filter Conditions¶
-
Data Sources: Include LOGs, APM, RUM, Events, and Audit Events.
-
Filter Conditions: Supports custom logic between conditions; multiple conditions can be added.
-
All Conditions: Only log data that meets all filter conditions will be saved for forwarding.
-
Any Condition: Any log data that meets any filter condition will be saved for forwarding.
-
Condition Operators are as follows:
| Condition Operators | Match Type |
|---|---|
| in, not in | Exact match, supports multiple values (separated by commas) |
| match, not match | Fuzzy match, supports regular expression syntax |
If no filter conditions are added here, it means all data will be saved.
Pipeline Processing for Forwarded Data
The central Pipeline script affects the filtering and final content of forwarded data. When creating a script, you can check "Enable Pipeline Processing for Forwarded Data": if checked, the data will be processed by the script before filtering and forwarding; if not checked, the original data will be forwarded directly.
3. Select Archive Type¶
To provide a more comprehensive data forwarding storage method, the system supports the following storage paths.
Guance: Matched log data will be saved to Guance's OSS, S3, or OBS object storage.
Note
-
The above archive types are available site-wide.
-
When selecting Guance as the data forwarding storage object, the minimum storage period for log data is 180 days by default, and once the rule is created, it cannot be canceled. Fees will be charged daily during the storage period. You can go to Manage > Workspace Settings > Change Data Storage Policy to modify it.
Storage Format¶
Choose the data storage format as needed:
-
JSON: Text format, the default storage type, easy to view and use immediately.
-
Parquet: Columnar storage format, forwarded data cannot be viewed back in Guance.
Note
- When forwarding data to Guance, only JSON storage format is supported.
Encrypted Storage¶
After enabling encrypted storage, the system will perform symmetric encryption on the forwarded data. If you need to query or view this data later, the system can restore the encrypted data to its original content for display.
What is Symmetric Encryption?
Symmetric encryption is an encryption method that uses the same key to encrypt and decrypt data, just like a key that can both lock and unlock the same lock.
For more details, refer to Symmetric Encryption.
4. Define Data Viewing Permissions¶
Set viewing permissions for forwarded data to enhance data security.
All members of the workspace can view forwarded data.
Specify the member roles that can view forwarded data.