Logs List¶

The log explorer is one of your core tools for log analysis and troubleshooting. Faced with a massive amount of log data collected and reported by Guance, you can achieve efficient log information management through multiple operations such as searching, filtering, and exporting.

View Modes¶

In the log explorer, data can be viewed and analyzed in various chart formats.

List¶

All Logs¶

Displays the raw log data that has been collected.

Clustering Analysis¶

The log explorer provides an efficient clustering feature that performs similarity analysis on logs based on the message field and automatically displays the most recent 50 logs. You can also customize the clustering fields. After selecting a time range in the time widget, the system will analyze 10,000 logs within that period and aggregate similar entries.

In the clustering analysis list, you can manage the data through the following actions:

• Click & to sort by document count (default descending order);

• Click , choose to display 1 line, 3 lines, 10 lines, or all content.

• Click , export all clustered log data.

Charts¶

Based on count, last, first, count_distinct calculation modes, data is filtered under the by condition:

• Top Lists
• Time Series Graphs
• Pie Charts
• Treemap Charts
• Grouped Table Charts

Log Color Highlighting¶

To help quickly locate key information in logs, the system uses color highlighting to display log content. When entering keywords in the search bar, only matched keywords will be highlighted.

Quick Filtering¶

Display Items¶

On the Stack Mode > Display Items page, two parts of fields are shown:

• Filter Fields: These are the fields displayed in quick filters;
• Optional Fields: These are all fields cached for the current data type.

You can perform the following actions:

• Search for fields; if no exact match is found in the query results, you can directly create and add them to "Filter Fields";
• Edit field aliases;
• Drag and drop to adjust field order;
• Delete fields;
• Set whether to display field aliases.

For more details, refer to Filtering.

Status Distribution Chart¶

Based on the selected time range, the system will automatically divide into multiple time points and display the number of different log statuses in the form of stacked bar charts, assisting in efficient statistical analysis. When filtering logs, the bar chart will synchronize and display the filtered results in real-time.

• You can hover over and export the chart, ultimately exporting it to dashboards, notes, or copying it to the clipboard;
• Customize the time interval selection.

Log Indexes¶

By setting up Multi-log Indexes, logs that meet the conditions are stored in different indexes respectively, and appropriate data storage strategies are chosen for each index, effectively saving log data storage costs.

After setup, you can switch between different indexes in the viewer to view corresponding log content.

Search and Filtering¶

In the log explorer's search bar, multiple search and filter methods are supported.

After entering search or filter conditions, you can preview the effects and copy these conditions to apply them to charts or query tools.

Manual Configuration¶

Click the toggle button on the right side of the search box to enter manual query input mode.

JSON Field Returns¶

DQL queries support extracting nested values from JSON fields in log data. You just need to add fields with the @ symbol in the DQL query statement, and the system will automatically recognize this configuration and display the extracted values as independent fields in the query results. For example:

• Normal Query:

• Expected query after extracting embedded fields:

In the log explorer, if you want to specify viewing extracted values from the JSON text of each log's message directly in the data list, add fields in the format @targer_fieldname in the display columns. As shown below, we add @fail_reason configured in the DQL query statement in the display columns:

Create Monitors¶

When filtering log data, if further alert monitoring of the filtered results is required, you can set up monitors with a single click. The system will automatically apply the selected index, data source, and search conditions, thus simplifying the configuration process.

Copy as cURL¶

In the log explorer, you can obtain log data via command-line. In the Settings on the right side of the log data list, click the Copy as cURL button to copy the corresponding cURL command. Pasting this command into the host terminal and executing it will retrieve log data that meets the filtering and search criteria for the current time period.

Example

After copying the cURL command line, as shown in the figure below: <Endpoint> needs to be replaced with the domain name, and <DF-API-KEY> should be replaced with the Key ID from API Management.

For more related parameter descriptions, refer to DQL Data Query.

For more API information, refer to Open API.

curl '<Endpoint>/api/v1/df/query_data?search_after=\[1680226330509,8572,"L_1680226330509_cgj4hqbrhi85kl1m6os0"\]&queries_body=%7B%22queries%22:\[%7B%22uuid%22:%222eb41760-cf6e-11ed-a983-7d559044c3fc%22,%22qtype%22:%22dql%22,%22query%22:%7B%22q%22:%22L::re(%60.*%60):(%60*%60)%7B+%60index%60+IN+\[%27default%27\]+%7D%22,%22highlight%22:true,%22limit%22:50,%22orderby%22:\[%7B%22time%22:%22desc%22%7D\],%22_funcList%22:\[\],%22funcList%22:\[\],%22disableMultipleField%22:false,%22disable_slimit%22:false,%22is_optimized%22:true,%22offset%22:0,%22search_after%22:\[1680226330509,8572,%22L_1680226330509_cgj4hqbrhi85kl1m6os0%22\],%22timeRange%22:\[1680187562081,1680230762081\],%22tz%22:%22Asia%2FShanghai%22%7D%7D\]%7D' \
- H 'DF-API-KEY: <DF-API-KEY>' \
- -compressed \
- -insecure

Besides this export path, you can also use other log data export methods.

Set Status Colors¶

The system has preset default colors for status values. If you need to customize the colors displayed in the viewer for different statuses, click Set Status Colors to modify.

Formatting Configuration¶

Through formatting configuration, you can hide sensitive log content, highlight important log content, or implement quick filtering by replacing log content.

1. Click the Settings in the top-right corner of the viewer list;
2. Click Formatting Configuration;

3. Add mapping rules, enter the following contents, and save:

   • Field: Specify the log field (e.g., content).
   • Match Method: Choose the match method (currently supports =, !=, match, not match).
   • Match Content: Input the content to match (e.g., DEBUG).
   • Display As Content: Input the replacement content to display (e.g., **).

Log Data Export¶

In logs, you can first filter out the required data, then export it as CSV, JSONL files, or export it to dashboards, notes.

If you need to export a specific log, open the log detail page and click in the top-right corner.

Single Log Row Expansion and Copy¶

• Click the button in the log entry to view the complete content of the log. If the log supports JSON format, it will be displayed in JSON format; otherwise, it will show normally.

• Click the button to copy the entire log content to the clipboard.

Multi-Line Log Browsing¶

In the log data list, the trigger time and content of each log are displayed by default. You can go to Viewer > Display Columns, and choose to display "1 Line", "3 Lines", "10 Lines", or "All Content" to view the full log information.