Datakit Tracing Data Structure¶

Brief¶

This article is used to explain the data structure of the mainstream Telemetry platform and the mapping relationship with the data structure of Datakit platform. Currently supported data structures: DataDog, Jaeger, OpenTelemetry, SkyWalking, Zipkin.

Data conversion steps:

1. External Tracing data structure access
2. Datakit Span transformation
3. span data operations
4. Line Protocol transformation

Datakit Point Protocol Data Structure¶

The Line Protocol data structure is a string composed of Name, Tags, Fields, Timestamp and delimiters (English commas, spaces), which is shaped like:

source_name,key1=value1,key2=value2 field1=value1,field2=value2 ts

Hereinafter referred to as dkproto

Span Type is the relative position of the current span in trace, and its value is described as follows:

• entry: the current api is the first call after the entry of the link into the service
• local: the current api is the api after the entrance and before the exit
• exit: the current api is the link's last call on the service
• unknown: the relative position state of the current api is not clear

Priority Rules samples priority rules for clients:

• PRIORITY_USER_REJECT = -1 User chooses to reject reporting
• PRIORITY_AUTO_REJECT = 0 Client sampler chooses to reject reporting
• PRIORITY_AUTO_KEEP = 1 Client sampler select report
• PRIORITY_USER_KEEP = 2 User chooses to report

Datakit Tracing Span Data Structure¶

TraceID    string                 `json:"trace_id"`
ParentID   string                 `json:"parent_id"`
SpanID     string                 `json:"span_id"`
Service    string                 `json:"service"`     // service name
Resource   string                 `json:"resource"`    // resource or api under service
Operation  string                 `json:"operation"`   // api name
Source     string                 `json:"source"`      // client tracer name
SpanType   string                 `json:"span_type"`   // relative span position in tracing: entry, local, exit or unknown
SourceType string                 `json:"source_type"` // service type
Tags       map[string]string      `json:"tags"`
Metrics    map[string]interface{} `json:"metrics"`
Start      int64                  `json:"start"`    // unit: nano sec
Duration   int64                  `json:"duration"` // unit: nano sec
Status     string                 `json:"status"`   // span status like error, ok, info etc.
Content    string                 `json:"content"`  // raw tracing data in json

Datakit Span is a data structure used internally by Datakit. The third-party Tracing Agent data structure is converted into a Datakit Span structure and sent to the data center.

Hereinafter referred to as dkspan

DDTrace Trace&Span Data Structure¶

DDTrace Trace Data Structure¶

DataDog Trace Structure

Trace: []*span

DataDog Traces Structure

Traces: []Trace

DDTrace Span Data Structure¶

OpenTelemetry Tracing Data Structure¶

When DataKit collects data sent from OpenTelemetry exporter: Otlp, the abbreviated raw data, after serialization by json, looks like this:

resource_spans:{
    resource:{
        attributes:{key:"message.type"  value:{string_value:"message-name"}}
        attributes:{key:"service.name"  value:{string_value:"test-name"}}
    }
    instrumentation_library_spans:{instrumentation_library:{name:"test-tracer"}
    spans:{
        trace_id:"\x94<\xdf\x00zx\x82\xe7Wy\xfe\x93\xab\x19\x95a"
        span_id:".\xbd\x06c\x10ɫ*"
        parent_span_id:"\xa7*\x80Z#\xbeL\xf6"
        name:"Sample-0"
        kind:SPAN_KIND_INTERNAL
        start_time_unix_nano:1644312397453313100
        end_time_unix_nano:1644312398464865900
        status:{}
    }
    spans:{
           ...
        }
}

The correspondence between resource_spans and dkspan in otel is as follows:

otel has some unique fields, but DKspan has no corresponding fields, so it is placed in the label and will only be displayed if these values are not 0, such as:

Jaeger Tracing Data Structure¶

Jaeger Thrift Protocol Batch Data Structure¶

Jaeger Thrift Protocol Span Data Structure¶

SkyWalking Tracing Data Data Structure¶

SkyWalking Segment Object Generated By Protocol Buffer Protocol V3¶

SkyWalking Span Object Data Structure in Segment Object¶

Zipkin Tracing Data Data Structure¶

Zipkin Thrift Protocol Span Data Structure V1¶

Zipkin Span Data Structure V2¶