Log Explorer¶

After log data is collected into Guance, you can view all the log content in the workspace through the Log Explorer in the Guance console.

Note: If the current logged-in account role has set Only display rules related to me in Logs > Data Access, the queried log content will be affected accordingly.

Modes¶

The Log Explorer supports three viewing modes, including:

• All Logs;
• Pattern;
• Chart Analysis.

All Logs¶

View and analyze based on the collected raw log data.

Pattern¶

Provide an efficient pattern feature, which performs similarity calculation and analysis on raw log data.

The system defaults to clustering logs by the message field, showing the latest 50 entries. Custom clustering is available for business needs. After setting a time range, it analyzes 10,000 logs, groups similar entries, and displays common features.

In the pattern list, the following operations are supported:

1. Click the icons & to sort the document quantity, with a default reverse order.

2. Click the button to choose the display of 1 line, 3 lines, 10 lines, and all.

3. Click the material-tray-arrow-up: button to export all clustered log data.

Chart Analysis¶

Under the mode, the system groups and statistically analyzes the original log data based on 1 or 3 tags, reflecting the distribution characteristics and trends of log data under different groups and times.

Four types of charts are available: time series charts, leaderboards, pie charts, and tree maps.

You can manage the chart analysis mode through the following operations:

1. Below the chart, filter queries based on various fields.

2. In the time series chart mode, select area charts, line charts and bar charts for display styles.

3. For the displayed clustered data, in the time series chart mode, you can choose slimit as 5/10/20/50/100; in the leaderboard/pie chart/tree map mode, you can choose limit to the maximum or minimum as 5/10/20/50/100.

4. Click the right button to choose to hide, bottom, or right legend.

5. Click the button to customize the color style of the legend display.

6. Click the button to choose to export the current chart as a CSV file, export to notes, dashboard, or directly copy the chart.

Query and Analysis¶

Log Statistics¶

Guance automatically divides several time points based on the selected time range and displays the number of different log statuses through a stacked bar chart, helping with statistical analysis. If logs have been filtered, the bar chart displays the results after filtering.

1. Hover the mouse over the chart and click the export button to export to a dashboard, notes, or clipboard.

2. Custom selection of time intervals.

Time Widget¶

Log explorers default to displaying log data from the last 15 minutes, and you can also customize the time range for data display.

Log Indexes¶

Exolorers can filter logs that meet the conditions and saving them in different log indexes, thus helping you save on log data storage costs by choosing different data storage strategies for log indexes.

After setting the index in Logs > Indexes, you can select different indexes in the explorer to view the corresponding log content.

For more details, see Log Indexes.

Search and Filtering¶

Multiple search methods and filtering methods are supported.

After entering the search or filter conditions, you can view the preview effect of the query. You can also copy the condition and use it directly for charts or query tools.

DQL Search¶

In the log explorer, you can switch to the DQL manual input query mode by clicking the switch button in the search box, and customize the input filter and search conditions.

• Filter conditions: Support any combination of and / or, using () parentheses to indicate the priority of executing the search and operators such as =, !=.

• Search conditions: Use DQL functions query_string() for string queries, such as entering message = query_string() to search log content.

For more DQL syntax, see DQL Definition.

Quick Filters¶

Support editing quick filters and adding new filter fields.

Note: If the values displayed in the quick filters are affected by sampling, the sampling rate is displayed, and users can temporarily turn off sampling.

Customize Display Columns¶

Explorers default to displaying the time and message fields, among which the time field is a fixed field and cannot be deleted. Hovering over a display column reveals the Settings button, which allows you to perform various column operations including sorting, moving columns, adding or removing columns and applying filters and groupings.

For more customization of display columns, see Display Column Description.

JSON Field Return¶

Guance DQL query supports extracting embedded values from the JSON fields of log data. By adding a field with the @ symbol in the DQL query statement, the system will recognize the configuration and display it as an independent field in the query results.

For example:

• Normal query:

• Expected extraction of embedded fields after query:

If you want to directly specify the value extracted from the message JSON text of each log in the data list, add a field in the display column in the format of @target_fieldname. In the figure below, we have added the @fail_reason that has been configured in the DQL query statement to the display column:

Create a Monitor¶

You can directly jump to the monitor creation page from the entry to quickly set up incident detection rules for log data.

For specific operations, see Create.

Copy as cURL¶

Note: Only Standard members and above can perform copy command line operations.

Support obtaining log data in the form of command lines. Click Copy as cURL to copy the cURL command line, go to the host terminal to execute the command, and obtain the log data related to the current time period under the current filter and search conditions.

Example

After copying the cURL command line, as shown in the figure below: <Endpoint> needs to be replaced with the domain name, and <DF-API-KEY> needs to be replaced with Key ID at API Management.

For more information on related parameters, see DQL Data Query.

For more information on the API, see Open API.

curl '<Endpoint>/api/v1/df/query_data?search_after=\[1680226330509,8572,"L_1680226330509_cgj4hqbrhi85kl1m6os0"\]&queries_body=%7B%22queries%22:\[%7B%22uuid%22:%222eb41760-cf6e-11ed-a983-7d559044c3fc%22,%22qtype%22:%22dql%22,%22query%22:%7B%22q%22:%22L::re(%60.*%60):(%60*%60)%7B+%60index%60+IN+\[%27default%27\]+%7D%22,%22highlight%22:true,%22limit%22:50,%22orderby%22:\[%7B%22time%22:%22desc%22%7D\],%22_funcList%22:\[\],%22funcList%22:\[\],%22disableMultipleField%22:false,%22disable_slimit%22:false,%22is_optimized%22:true,%22offset%22:0,%22search_after%22:\[1680226330509,8572,%22L_1680226330509_cgj4hqbrhi85kl1m6os0%22\],%22timeRange%22:\[1680187562081,1680230762081\],%22tz%22:%22Asia%2FShanghai%22%7D%7D\]%7D' \
- H 'DF-API-KEY: <DF-API-KEY>' \
- -compressed \
- -insecure

In addition to this export path, you can also use other log data export methods.

Status Colors¶

Note: Only Administrator members and above can change status colors.

Guance has set default system colors for status values. You can modify the colors displayed in the explorer for corresponding data under different status.

Format¶

Note: Only Administrator members and above can perform format for explorers.

Hide sensitive log data content or highlight the log data content you need to view, and quickly filter by replacing the original log content.

Click on Format to add a mapping. Enter the following content and click save to replace the original log content containing "DEBUG" with the format you want to display.

• Field: such as content;
• Matching method: such as match (currently supports =, !=, match, not match);
• Matching content: such as DEBUG;
• Display as content: such as ******.

Export¶

Filter the log data you want, and then export it for viewing and analysis through . You can export to CSV files or dashboards and notes.

If you need to export a specific log entry, open the log detail page and click the icon in the upper right corner.

Color Highlighting¶

To help you quickly obtain key log data information, Guance uses different colors to highlight different content in the logs, divided into light and dark theme colors.

Note: If you search for logs in the search bar, the returned list will only retain the highlighting display of the matched keywords.

Single Line Expansion Copy¶

Click on a log entry to expand and view the entire content of the log;

Click the button to copy the entire log entry. When the log content is expanded, if JSON display is supported, the log will be displayed in JSON format; if not supported, the log content will be displayed normally.

Multi-line Browsing¶

The log data list in Guance defaults to showing you the trigger time and content of the log. You can choose to display the log in "1 line," "3 lines," "10 lines," and all in the explorer Display Columns to view the complete log content.

Log Details¶

Click on the log list to pull out the current log's detail page to view detailed information of the log, including the time the log was generated, the host, source, service, content, extended fields, view context, etc.

View Complete Logs¶

When logs are reported to Guance, if the data exceeds 1M in size, they will be split according to the 1M standard. For example, a 2.5M log will be divided into 3 pieces (1M/1M/0.5M). You can view the integrity of the split logs based on the following fields:

On the log detail page, if the current log is split into multiple pieces, the View Complete Logs button will be displayed in the upper right corner. Click it and open a new page and list all related logs according to the log split order. The page will also mark the previously selected log with color to help locate upstream and downstream.

Log Context¶

The context query feature of the log service can help you use the timeline to trace the log records before and after a specific exception log. Through the visualization of log context information in Guance, it can effectively assist you in troubleshooting and problem localization, allowing for timely response.

On the log detail page, you can directly view the context logs of the data content; click to open a new page of context logs.

L::RE(`.*`):(`message`) { `index` = 'default' and `host` = "ip-172-31-204-89.cn-northwest-1" AND `source` = "kodo-log" AND `service` = "kodo-inner" AND `filename` = "0.log" and `log_read_lines` >= 1354140 and `log_read_lines` <= 1354200}  sorder by log_read_lines

Detail Page¶

Click to be redirected to the detail page:

You can manage all current data with the following operations:

1. Enter text in the search box to search and locate data;
2. Click the side button to change the system's default selection of automatic line wrapping, and choose Content Overflow to display each log entry as one line, which can be viewed by sliding left and right as needed.

Attribute Fields¶

When you click on the attribute fields for quick filtering and viewing, you can view the host, process, link, and container data related to the log.

Log Content¶

• Log content automatically displays two viewing modes: JSON and text, based on the message type. If a log does not have a message field, the log content section will not be displayed. Log content can be expanded or collapsed, with the default being the expanded state. When collapsed, only one line of height is shown.

• For logs with source = bpf_net_l4_log, two viewing modes are automatically displayed: JSON and message. The message mode displays information such as the client, server, and time, and supports switching between absolute and relative time views. The default display is absolute time, and this setting will be saved to the local browser after switching.

Attributes¶

• In the search bar, you can quickly search and locate by entering the field name or value.

• After checking the field alias, you can view it after the field name.

• Hover over an extended field and click the dropdown icon to choose to filter field values, inverse filter field values, add to display columns, and copy for that field.

Associated Analysis¶

Bind Inner Views¶

Guance supports setting up the binding or removing of inner views (user views) to the log detail page.