Datakit Tracing Data Structure

Introduction¶

This document explains the data structures of mainstream Telemetry platforms and their mapping relationships with the Datakit platform data structure. Currently supported data structures: DataDog/Jaeger/OpenTelemetry/SkyWalking/Zipkin/PinPoint

Data transformation steps:

1. External Tracing data ingestion, where data is received via multiple protocols and then deserialized.
2. Deserialized objects are converted to Line Protocol (line protocol format).
3. Span data operations include: sampling, filtering, adding specific tags, etc.

Datakit Point Protocol Data Structure¶

• Tags

• Field

The span_type indicates the relative position of the current Span within the Trace. The values are as follows:

• entry: The current API is the entry point, i.e., the first call after entering the service.
• local: The current API is between the entry and exit points.
• exit: The current API is the last call in the trace on the service.
• unknown: The relative position of the current API is unclear.

The priority represents the client-side sampling priority rules:

• PRIORITY_USER_REJECT = -1 User chooses to reject reporting
• PRIORITY_AUTO_REJECT = 0 Client sampler chooses to reject reporting
• PRIORITY_AUTO_KEEP = 1 Client sampler chooses to report
• PRIORITY_USER_KEEP = 2 User chooses to report

OpenTelemetry Tracing Data Structure¶

When Datakit collects data sent from the OpenTelemetry Exporter (OTLP), the simplified raw data serialized in JSON format is as follows:

resource_spans:{
    resource:{
        attributes:{key:"message.type"  value:{string_value:"message-name"}}
        attributes:{key:"service.name"  value:{string_value:"test-name"}}
    }
    instrumentation_library_spans:{instrumentation_library:{name:"test-tracer"}
    spans:{
        trace_id:"\x94<\xdf\x00zx\x82\xe7Wy\xfe\x93\xab\x19\x95a"
        span_id:".\xbd\x06c\x10ɫ*"
        parent_span_id:"\xa7*\x80Z#\xbeL\xf6"
        name:"Sample-0"
        kind:SPAN_KIND_INTERNAL
        start_time_unix_nano:1644312397453313100
        end_time_unix_nano:1644312398464865900
        status:{}
    }
    spans:{
           ...
        }
}

The correspondence between resource_spans in OpenTelemetry and DKProto is as follows:

OpenTelemetry has some unique fields that do not have corresponding fields in DKProto, so they are placed in the tags. These values are only displayed when non-zero, such as:

Jaeger Tracing Data Structure¶

Jaeger Thrift Protocol Batch Data Structure¶

Jaeger Thrift Protocol Span Data Structure¶

SkyWalking Tracing Data Data Structure¶

Segment Object Generated By Protobuf Protocol V3¶

SkyWalking Span Object Data Structure in Segment Object¶

Zipkin Tracing Data Data Structure¶

Zipkin Thrift Protocol Span Data Structure V1¶

Zipkin Span Data Structure V2¶