Trace Details¶

In the trace explorer, clicking on any trace allows you to view detailed information about the trace, including relative time, duration, HTTP method, URL, status code, TraceId, flame graph, Span list, service call relationships, and associated logs, hosts, metrics, and network information. For frontend applications (such as browsers), you can also view the distribution of request durations, including queue time, first packet time, and download time.

Note

The RUM SDK must be version 2.2.10 or higher for this data display to appear. If there are cross-origin issues, you need to adjust the header configuration.

For more details, refer to Web Application Integration.

The flame graph/Span list/waterfall chart displays up to 10,000 Spans. You can use the offset setting to view Spans that are not displayed:

Flame Graph¶

The flame graph clearly shows the flow and execution time of each Span. On the right side, you can view the service list and response times. Clicking on a Span allows you to view its JSON content in the trace details, and you can zoom in using the mouse scroll wheel to view specific information.

For more details, refer to Effectively Using Flame Graphs to Analyze Trace Performance.

Flame Graph Trace DescriptionExecution Time Ratio Calculation ExplanationAsynchronous Calls

From the flame graph above, you can see that this call trace includes two services: CloudCare and Mysql. The trace starts with a POST request initiated by the CloudCare service, then executes ActionResource.executeAction, and finally executes Mysql statements. Throughout the execution of ActionResource.executeAction, multiple Mysql statements are executed. The execution times for CloudCare and Mysql are calculated as follows:

Execution time for the CloudCare service = D1+D2+D3+D4+D5+D6+D7+D8+D9+D10+D11
Execution time for the Mysql service = span2+span3+......+span11

Specific execution statements and execution time details can be referenced from the Span list.

The service execution time ratio in the flame graph refers to the proportion of time each service takes in the total time for this call trace. In the example below, this call trace includes two services: CloudCare and Mysql, with execution time ratios of 42.37% and 57.63%, respectively. CloudCare service has 2 Spans, while Mysql service has 10 Spans.

Calculation method for Mysql service execution time ratio: sum of all span execution times / total duration of the current call trace.

Explanation of the calculation method: In the figure below, the Mysql service has a total of 10 Spans. You can click on each Span to get its execution time. From the figure, you can see that the execution time for this Span is 5.08ms. Similarly, obtain the execution times for the remaining 9 Spans and add them together.

Calculation method for CloudCare service execution time ratio: (total duration of the current call trace - mysql service execution time) / total duration of the current call trace.

Explanation of the calculation method: In the figure below, the CloudCare service spans the entire current call trace. The remaining time, excluding the execution time of the mysql service, is the execution time for the CloudCare service (see the marked red line section for execution time). The execution time ratio can also be directly viewed via the Span list for each Span's execution time and execution time ratio.

In the flame graph, whether the service calls are synchronous or asynchronous, it can clearly track the performance data details of each trace. For example, through the flame graph, you can clearly see which requests were performed asynchronously, along with their start time, end time, and total time spent.

Span List¶

Displays all Span resource names, quantities, durations, execution times, and proportions within the trace.
You can input a resource name or Span ID to search and match.
Clicking any Span allows you to view its JSON content in the trace details and switch to synchronized display in the flame graph.
If there is an error, an error message will be displayed.
Clicking Error Spans, directly displays the results of the filtered hits.

Waterfall Chart¶

Displays the parent-child relationships between various resources.

The waterfall chart displays Span data according to the order of start times. The left-hand list shows Span data and the execution time proportions for each resource. The right-hand side displays the waterfall chart based on time sequence.

Input the resource name or Span ID corresponding to the Span to perform a search match;
Click to toggle the format of execution time;
Click Error Spans to directly display the results of the filtered hits.

Service Call Relationships¶

Displays the call relationships and call counts between services, searchable and filterable by service, resource, or Span ID.

The system displays the color of services here based on the error results corresponding to the services counted in the trace details > flame graph. Red indicates the presence of errors.

If you have configured a service binding relationship in the user view, such as a service:mysql relationship, clicking the service card allows you to quickly view the associated view. Clicking will redirect you to the detail page.

Shortcut Operations¶

Operation	Description
Full-screen View/Restore Default Size	You can click the full-screen view icon at the top-right corner of the trace details to expand horizontally and view the trace flame graph; click the restore default size icon to return to the details page.
Locate to Current Span	Click to zoom in and locate the current Span
Expand/Collapse Minimap	You can click the expand/collapse minimap icon on the left side of the trace details to quickly view the flame graph by selecting intervals, dragging, or scrolling on the minimap.
View Global Trace	You can click the view global Trace icon on the left side of the trace details to view the global trace in the flame graph.
Double-click Span	In the flame graph, double-clicking a Span displays it enlarged in the middle, allowing you to quickly locate and view its context-related Spans.
Click Service Name	Highlights the corresponding Span. Clicking the service name again restores the default selection of all Spans. You can quickly filter and view Spans corresponding to the service by clicking the service name.

Extended Attributes¶

In the search bar, you can input field names or values to quickly search and locate;

After checking field aliases, they can be viewed after the field name. Select as needed.

In the trace details page, you can view relevant field attributes of the current trace under Extended Attributes:

Field	Attribute
Filter Field Values	Adds the field to the Explorer to view all data related to the field. Can be filtered and viewed in the trace Explorer. See Figure One.
Reverse Filter Field Value	Adds the field to the Explorer to view data other than the field.
Add to Display Columns	Adds the field to the Explorer list for viewing.
Copy	Copies the field to the clipboard.

Some Fields Do Not Support Filtering Aggregation Logic

Guance contains some fields using full-text indexing. These fields do not support filtering aggregation logic. The full-text index field range is as follows:

Category	Field
Objects, Resource Catalog	`message`
Logs, Backup Logs	`message`
Security	`message` / `title`
Network	`message`
Trace	`error_message` / `error_stack`
Events	`message` / `title` / `df_message` / `df_title`
RUM Errors	`error_message` / `error_stack`
RUM Long Tasks	`long_task_message` / `long_task_stack`

Figure One

Error Details¶

On the trace details page, if there are error traces, you can view the related error details.

For more analysis of error traces, refer to Error Tracking.

Service Context¶

By obtaining object classifications from the infrastructure resource catalog and selecting the latest object based on create_time, you can quickly view the runtime information, service dependencies, and integration information for the current service.

Correlation Analysis¶

Correlated LogsCode HotspotsCorrelated HostsCorrelated ContainersCorrelated PodsCorrelated Networks

You can view logs associated with the current trace (correlation field: trace_id) through logs. You can customize the display columns, and if you need to view more detailed log content, you can click the log content to go to the log details page or click the jump button to open the log page.

If you have administrator-level permissions or above, you can customize correlation fields. Click the settings button to the right of the correlation field, select the fields you need to correlate in the pop-up dialog box, and complete the configuration by supporting manual input, drag-and-drop order operations, etc.

Note: Custom correlation fields for logs and custom correlation fields for service lists affect each other. If custom fields are configured in the service list, they will be displayed synchronously here.

When the application uses ddtrace collectors and simultaneously enables APM trace tracking and Profile performance tracking data collection, Guance provides Span-level correlated viewing and analysis. On the trace details page, you can click Code Hotspots below the flame graph to view the code hotspots associated with the current trace, including execution time, methods, and execution time proportions.

Clicking View Profile Details allows you to go to the Profile details page to view more associated code.

On the trace details page, you can view the metric views and property views of related hosts (correlation field: host).

Metric View: You can view the performance metric status of related hosts from 30 minutes before the trace ends to 30 minutes after the trace ends, including CPU, memory, and other performance metric views of related hosts.

Property View: Helps you trace back to the actual state of host objects when the trace was generated, supports viewing the latest object data produced by related hosts within the corresponding time period, including basic information about the host, integration runtime conditions. If cloud host collection is enabled, you can also view cloud vendor information.

Note: Guance defaults to saving the historical data of host objects for the last 48 hours. If no historical data for the host corresponding to the current trace time is found, you will not be able to view the property view of the associated host.

On the trace details page, you can view the metric views and property views of related containers (correlation field: container_name).

Metric View: Supports viewing the performance metric status of related containers from 30 minutes before the trace ends to 30 minutes after the trace ends, including container CPU, memory, and other performance metric views.
Property View: Helps you trace back to the actual state of container objects when the trace was generated, supports viewing the latest object data produced by related containers within the corresponding time period, including basic information and attribute information about the containers.

On the trace details page, you can view the property views and metric views of related Pods (correlation field: pod_name).

Metric View: Supports viewing the performance metric status of related Pod containers from 30 minutes before the trace ends to 30 minutes after the trace ends, including container CPU, memory, and other performance metric views.
Property View: Helps you trace back to the actual state of Pod objects when the trace was generated, supports viewing the latest object data produced by related Pod containers within the corresponding time period, including basic information and attribute information about the containers.

Guance supports you to view multi-dimensional network topology maps and overview data including Host, Pod, Deployment, and Service in Network.

Matching Fields:

To view related networks on the details page, corresponding matching fields need to be configured during data collection; otherwise, you cannot match and view associated network views on the details page.

Host: Matching field host.
Pod:

Matching Field Priority
namespace、pod_name
namespace、pod
pod_name
pod

Deployment:

Matching Field Priority
namespace、deployment_name
namespace、deployment
deployment_name
deployment

Service:

Matching Field Priority
namespace、service_name
namespace、service

Note:

If Host, Pod, Deployment, and Service matching fields are found simultaneously, network data will be displayed in this order upon entering the details page;
If no matching fields are found, it will show gray at the bottom, and clicking will prompt No Network View Matched.