Data Forward¶

Guance offers the functionality to save logs, traces and user access data to the object storage of Guance and forward it to external storage. You are free to choose the storage objects and manage data forwarding flexibly.

On the data forwarding page, you can quickly query stored data (including Guance backup logs, AWS S3, Huawei Cloud OBS, Alibaba Cloud OSS, and Kafka message queues) by setting query time and data forwarding rules. You can also view the historical backup logs of Guance and SLS Query Logstore data.

Precondition¶

Only Guance Commercial Plan users can use the data forwarding function, Experience Plan users need to upgrade to the Commercial Plan.

Create Rules¶

On the Data Forward page, click Forward rules > Create.

Note: After creating the data forwarding rule, the rule validation will be executed every 5 minutes.

Enter Rule Name¶

In the pop-up dialog box, enter a name to add a new rule.

Field	Description
Rule Name	Limit the input to a maximum of 30 characters.
Include extend fields	If the option is selected, the entire log data that meets the criteria will be forwarded. The application performance and user access data will be forwarded by default, regardless of this option. If multiple data forwarding rules are created, the rule that includes extended fields will be prioritized. In other words, if different rules match the same data, the entire log data will be displayed according to the logic of synchronously including extended fields.

Define Filter Conditions¶

Data Source: Logs, Trace and RUM.

Filtering Conditions: Support custom logical operations between conditions. You can choose All Conditions or Any Condition:

• All Conditions: Only log data that meets all filtering conditions will be saved to data forwarding.
• Any Condition: Log data that meets any of the filtering conditions will be saved to data forwarding.

Note: Not adding any filtering conditions means saving all log data. You can add multiple filtering conditions.

The operators for conditions are shown in the table below:

Select Archive Type¶

To provide a more comprehensive data forwarding and storage method, Guance supports the following five storage paths:

• Guance;
• AWS S3;
• Huawei Cloud OBS;
• Alibaba Cloud OSS;
• Kafka Message Queue.

Note: All five types of archives are open to the entire site.

Guance¶

When the data forwarding storage object is set to Observation Cloud, the matched log data will be saved in the OSS, S3 and OBS object storage on Guance.

The minimum storage duration for log data under this rule is 180 days by default. Once the rule is created, it cannot be canceled, and fees will be charged daily during the storage period. You can go to Management > Settings > Change Data Storage Policy to modify the data forwarding storage policy.

AWS S3¶

I. Select the archival type as AWS S3, which means that the matched log data will be saved in the S3 object storage;
II. Choose the access type:

III. Click Confirm to create successfully.

Note: If there are changes in the archive type information, a confirmation dialog will pop up to confirm the creation rule.

Huawei Cloud OBS¶

I. Select the Huawei Cloud OBS as the archive type, which means that the matched logs will be automatically forwarded to external OBS.

II. In order to configure the access authorization for Huawei Cloud resources, you need to use the exclusive Huawei Cloud account ID f000ee4d7327428da2f53a081e7109bd provided by Guance. Go here to add cross-account access authorization policies.

Note: The account ID for overseas sites is different from that of the Chinese site, please make a distinction:

III. Select Region.

IV. In the Bucket, enter the name of your bucket in Huawei Cloud.

V. Click Confirm to create successfully.

Alibaba Cloud OSS¶

I. Select the Alibaba Cloud OSS as the archive type, which means that the matched log data will be saved to Alibaba Cloud OSS object storage.
II. Choose access type:

III. Click Confirm to create successfully.

Note: If there are changes in the archive type information, a confirmation dialog will pop up to confirm the creation rule.

Kafka Message Queue¶

I. Server Address: Host:Port; multiple nodes are separated by commas.

II. Topic: the name of the topic.

III. Security protocol:

On the Kafka side, SASL can use either the PLAINTEXT or SSL protocol as the transport layer. Correspondingly, you can use SASL_PLAINTEXT or SASL_SSL security protocol. If using SASL_SSL security protocol, SSL certificate must be configured.

Click Test Connection. If the above information meets the requirements, a successful connection test will be prompted.

If the test fails, you need to confirm:

• Whether the address is correct;
• Whether the message topic name is correct;
• Whether the SSL certificate is correct;
• Whether the username is correct;
• Whether the password is correct.

V. Click Confirm to create successfully.

View Forwarding Rules¶

After the rule is created, it will automatically enter the forwarding rule list:

You can search by entering the rule name.

You can choose to enable or disable the current rule.

Click the , edit button and button on the right side of the rule to perform the corresponding operation.

Note: Data that has been forwarded will not be deleted after the rule is deleted, but no new forwarding data will be generated.

You can select multiple rules for batch operations.

Forwarding Rule Explorer¶

Go back to the Data Forward page, which defaults to the Forward Rules tab. First, select a rule from the dropdown menu. You can customize the time range for the query, selecting multiple dates and defining a start and end time. The time will be accurate to the hour:

Note:

• You can enter keywords to search for matching data.
• The time control is initially empty, and you can select the hour after selecting a date. Clickable hour options will be listed based on the forwarding rule.
• If you select a future time range, the system will automatically correct it to the current date.

• Guance will retrieve file search matching data in batches based on the selected time. Each batch returns 50 pieces of data. If no data is found in the first query, or the returned data does not meet the requirement of 50 pieces per page, you can manually click Continue Query until the scan is complete.
• Since the retrieved data is in a disordered state, you can sort the data based on the listed time range. This action will not affect the data query results.

Under Index, you can view Guance historical backup logs and SLS Query Logstore data:

For specific operations of Explorer, see The Power of the Explorer.

More Readings¶