Trace Details¶
In the Trace Explorer, click any trace to view its details, including relative time, duration, HTTP method, URL, status code, TraceId, flame graph, span list, service call relationships, and associated logs, hosts, metrics, and network information. For frontend applications (e.g., browsers), you can also view the request time distribution, including the time proportions for queuing, first byte, and download.
Note
- The RUM SDK must be version 2.2.10 or higher to see this data. If cross-origin issues exist, header configuration adjustments are needed.
For more details, refer to Web Application Integration.
- The flame graph/span list/waterfall chart displays up to 10,000 spans. You can use the offset setting to view spans not shown:
Flame Graph¶
The flame graph clearly shows the flow and execution time of each span. The right side displays the service list and response time. Click a span to view its JSON content in the trace details. Use the mouse wheel to zoom in for specific information.
For more details, refer to Analyzing Trace Performance with Flame Graphs.
From the flame graph above, this trace involves two services: CloudCare and Mysql. The trace starts with a POST request to the CloudCare service, then executes ActionResource.executeAction, and finally executes Mysql statements. During the execution of ActionResource.executeAction, multiple Mysql statements are executed. The execution times for CloudCare and Mysql are calculated as follows:
- CloudCare service execution time = D1+D2+D3+D4+D5+D6+D7+D8+D9+D10+D11
- Mysql service execution time = span2+span3+......+span11
For details on the specific statements executed and their execution times, refer to the Span List.
The service execution time percentage in the flame graph represents the ratio of each service's execution time to the total trace duration. In the example below, the trace involves two services: CloudCare and Mysql, with execution time percentages of 42.37% and 57.63% respectively. The CloudCare service has 2 spans, and the Mysql service has 10 spans.
- Mysql service execution time percentage calculation: Sum of all span execution times / Total trace duration.
Calculation explanation: The Mysql service has 10 spans in the image below. Click each span to get its execution time. From the image, this span's execution time is 5.08ms. Similarly, obtain and sum the execution times of the remaining 9 spans.
- CloudCare service execution time percentage calculation: (Total trace duration - Mysql service execution time) / Total trace duration
Calculation explanation: The CloudCare service spans the entire trace duration in the image below. Except for the Mysql service execution time, the remaining time is the CloudCare service execution time (see the red-lined execution time). The execution time percentage can also be viewed directly in the Span List for each span's execution time and percentage.
In the flame graph, whether service calls are synchronous or asynchronous, every detail of the trace performance data can be clearly tracked. For example, the flame graph clearly shows which requests are asynchronous, their start time, end time, and total duration.
Span List¶
- Displays all spans in the trace, including resource name, count, duration, execution time, and percentage.
- Search by resource name or Span ID.
- Click any span to view its JSON content in the trace details; the flame graph will sync to show the selected span.
- Errors are highlighted with error prompts.
- Click Error Spans to directly display filtered error spans.
Waterfall Chart¶
Shows the parent-child relationships between resources.
The waterfall chart displays span data in chronological order of start time. The left list shows span data with execution time percentages. The right side shows the waterfall chart in chronological order.
- Search by resource name or Span ID.
- Collapse or expand resources on the left with one click.
- Click to switch the execution time format.
- Click Error Spans to directly display filtered error spans.
Service Call Relationships¶
Provides an overview of call relationships and call counts between different services, as well as execution time, average duration, and call count for individual services.
Click any service node to view its details:
-
Service Operations and Resources: Lists specific operations (e.g., API interfaces, methods) and related resources for the service.
-
Error Status Tracking:
-
If the service has errors, error details are displayed directly.
-
Click to view the service's parent spans to trace upstream call sources.
-
Note
The system displays the service color here based on the error results for the corresponding service counted in the Trace Details > Flame Graph.
Quick Actions¶
Action |
Description |
|---|---|
| Full Screen View/Restore Default Size | Click the full-screen icon in the upper right corner of the trace details to horizontally expand the flame graph. Click the restore default size icon to return to the details page. |
| Locate Current Span | Click to zoom the view and locate the current span. |
| Expand/Collapse Mini Map | Click the expand/collapse mini map icon on the left side of the trace details. Use the mini map to select intervals, drag, and scroll for quick flame graph navigation. |
| View Global Trace | Click the view global trace icon on the left side of the trace details to view the global trace in the flame graph. |
| Double-Click Span | Zooms in on the span in the center of the flame graph, allowing you to quickly locate and view its contextually related spans. |
| Click Service Name | Highlights the corresponding spans. Click the service name again to restore the default selection of all spans. Use this to quickly filter and view spans for a specific service. |
Extended Attributes¶
-
In the search bar, enter a field name or value to quickly search and locate.
-
After checking the field alias, it will be displayed next to the field name. You can choose as needed.
-
On the trace details page, you can view the relevant field attributes of the current trace under Extended Attributes:
Field |
Attribute |
|---|---|
| Filter Field Value | Adds this field to the explorer to view all data related to this field. You can filter and view the trace list related to this field in the trace explorer. See Figure 1. |
| Invert Filter Field Value | Adds this field to the explorer to view data excluding this field. |
| Add to Display Columns | Adds this field to the explorer list for viewing. |
| Copy | Copies the field value to the clipboard. |
Some fields do not support filtering, aggregation, and related logic
Guance has some fields that use full-text indexing. These fields do not support filtering, aggregation, or related logic. The full-text indexed fields are as follows:
| Category | Fields |
|---|---|
| Object, Resource Catalog | message |
| Log, Backup Log | message |
| Security | message / title |
| Network | message |
| Trace | error_message / error_stack |
| Event | message / title / df_message / df_title |
| RUM Error | error_message / error_stack |
| RUM Long Task | long_task_message / long_task_stack |
Figure 1
Error Details¶
On the trace details page, if errors exist in the current trace, check the error span filter in the sidebar. The system will automatically highlight all error spans. Click any highlighted span to view its associated error stack, logs, and other detailed information.
For more details, refer to Error Tracking.
Service Context¶
By obtaining object classifications under the infrastructure resource catalog and selecting the latest object based on create_time for display, quickly view the current service's runtime information, service dependencies, and integration information.
Correlation Analysis¶
You can view all log data associated with all services in the trace through Logs (correlation field: trace_id). You can customize the display columns. To view more detailed log content, click the log content to jump to the log details page, or click the jump button to open it in the Logs page.
If you have Administrator or higher permissions, you can customize the correlation fields. Click the settings button next to the correlation field, select the fields to correlate in the pop-up dialog, support manual input, drag-and-drop ordering, etc., and confirm to complete the configuration.
Note: Custom fields for correlated logs and custom fields for Service List correlation analysis configuration affect each other. If custom fields are configured in the Service List, they are displayed here as well.
When the application uses the ddtrace collector with both APM trace collection and Profile performance trace data collection enabled, the system provides span-level correlation to view code hotspots, including execution time, methods, and execution time percentage.
Click View Profile Details to jump to the Profile details page and view more associated code.
On the trace details page, you can view related host metric views and attribute views through Hosts (correlation field: host).
- Metric View: View the performance status of related hosts from 30 minutes before the trace ended to 30 minutes after the trace ended, including CPU, memory, and other performance metric views for the related hosts.
- Attribute View: Helps you回溯 the actual state of the host object when the trace was generated. It supports viewing the latest piece of object data generated by the related host within the corresponding time, including basic host information and integration running status. If cloud host collection is enabled, cloud vendor information can also be viewed.
Note: Guance retains host object history data for the last 48 hours by default. If no host history data corresponding to the trace time is found, you will not be able to view the attribute view of the correlated host.
On the trace details page, you can view related container metric views and attribute views through Containers (correlation field: container_name).
-
Metric View: Supports viewing the performance status of related containers from 30 minutes before the trace ended to 30 minutes after the trace ended, including container CPU, memory, and other performance metric views.
-
Attribute View: Helps you回溯 the actual state of the container object when the trace was generated. It supports viewing the latest piece of object data generated by the related container within the corresponding time, including basic container information and attribute information.
On the trace details page, you can view related Pod attribute views and metric views through Pods (correlation field: pod_name).
-
Metric View: Supports viewing the performance status of the related container Pod from 30 minutes before the trace ended to 30 minutes after the trace ended, including container CPU, memory, and other performance metric views.
-
Attribute View: Helps you回溯 the actual state of the container Pod object when the trace was generated. It supports viewing the latest piece of object data generated by the related container Pod within the corresponding time, including basic container information and attribute information.
Guance supports viewing network topology maps and overview data across multiple dimensions including Host, Pod, Deployment, and Service in Network.
Matching Fields:
To view the related network in the details page, the corresponding correlation fields must be matched, meaning the appropriate field tags need to be configured during data collection. Otherwise, the correlated network view cannot be matched and viewed in the details page.
-
Host: Match field
host. -
Pod:
| Matching Field Priority |
|---|
| namespace, pod_name |
| namespace, pod |
| pod_name |
| pod |
- Deployment:
| Matching Field Priority |
|---|
| namespace, deployment_name |
| namespace, deployment |
| deployment_name |
| deployment |
- Service:
| Matching Field Priority |
|---|
| namespace, service_name |
| namespace, service |
Note:
- If correlation fields for Host, Pod, Deployment, and Service are all found, the network data is displayed in this order when entering the details page.
- If no correlation fields are found, it is displayed in gray at the end, with a prompt No network view matched.