Added an HTTP concurrency limit before reading the request body, reusing kodo_queue.workers (DW_KODO_QUEUE_WORKERS, default 256). When the limit is reached, up to 128 additional requests can wait for at most 1s. This limit remains in effect when the Kodo dispatch queue is disabled; requests without a body are not counted (#115)
Added dataway_http_concurrency_in_flight, dataway_http_concurrency_waiting, dataway_http_concurrency_requests_total, and dataway_http_concurrency_wait_seconds metrics to observe the number of in-flight requests, waiting requests, admission results, and waiting time (#115)
Fixed a compatibility issue with Langfuse SDK 4.x in /api/public/ingestion when the upstream succeeds but returns an empty response body: now returns successes/errors JSON for each event ID in the request batch and preserves the upstream JSON response content when present (#116)
When the HTTP concurrency waiting queue is full or the wait times out, registered DataKit-compatible endpoints return 503, other endpoints return 429, and both include Retry-After: 1; rejected requests are not written to the disk cache (#115)
When an HTTP/1.x request is rejected by the concurrency limit, reading of the remaining body is stopped and the connection is closed after the response is returned, avoiding waits for incomplete uploads; clients must establish a new connection when retrying (#115)
Added the /v2/tail_sampling zstd reporting endpoint, and made it compatible with zstd packets sent by DataKit 2.10 via /v1/tail_sampling without a compression negotiation header (#110)
Added bounded batch sending and failure retry for tail-sampling retained data, changed memory overflow spill-to-disk to asynchronous processing, and improved restart recovery and exit cleanup mechanisms (#110)
Tail sampling now extracts trace_id after decompression and recomputes predicates; gzip compression is used when sending batches downstream to reduce request count and transfer overhead (#110)
Improved runtime metric coverage for tail sampling, including queue submission results, waiting time, disk depth, and batch processing status (#110)
Set the token label in metrics to a fixed value of redacted and no longer retain original token fragments, avoiding credential leakage and high-cardinality time series (#90)
Masked aggregation, tail sampling, and multi-backend forwarding errors so that backend URLs, response content, and authentication information are no longer exposed (#90)
Optimized the trace_id extraction path in tail sampling to avoid repeated payload traversal when RawGroupId is available, while keeping sampling behavior unchanged (#108)
Upgraded cliutils: capacity metrics are automatically initialized when the disk cache is enabled, and shutdown and file-lock release behavior was improved (#111)
Unified token validation and passing semantics for the Aggregate and Tail Sampling special APIs (#88)
Isolated the invalid-token negative cache by token, write route, and main/headless mode to prevent false blocking across scenarios (#106)
Fixed concurrent access issues in the failure cache and tail-sampling disk fallback path to avoid data races during disk queue initialization, reading, and closing (#111)
Isolated the pprof and Prometheus HTTP muxes; pprof is disabled by default, and when enabled it only listens on loopback, with limits on diagnostic request duration and concurrency (#89)
Masked tokens, credentials, global labels, and rule contents in request forwarding, cache replay, aggregation, tail sampling, and Sinker logs (#104)
Fixed system Workspace URL detection and added regression coverage for token injection in normal and cascading modes (#104)
Corrected the main HTTP server request header timeout, Gin mode, IP blacklist configuration example, and API wrapper error handling (#104)
Pinned the Go and golangci-lint download archives used by CI, verified SHA-256 checksums per architecture, and removed the direct execution path of remote installation scripts (#101)
Updated the default DataWay dashboard with additional monitoring for queues, tail sampling, Sinker cache, and runtime (#85)
The new Sinker cache stores data based on the request's hash fingerprint (16 bytes) rather than the full request characteristics. It uses two-way hashing to minimize hash collisions; theoretically, the collision probability is n/2^128, where n is the number of cached keys
The Sinker cache now has TTL and capacity mechanisms to further bound memory usage: the TTL mechanism cleans up inactive cache entries, while the capacity mechanism ensures that the number of cache keys does not exceed the specified limit. The TTL and capacity limits further reduce the hash collision probability mentioned above
Added a configuration option to optimize the HTTP headers returned by DataWay, reducing DataWay public network traffic (#65)
Added a time synchronization endpoint so DataKit can obtain more accurate Unix timestamps (#40)
Sinker:
Filter conditions now support nil checks, that is, determining whether a specific field exists (#41)
Added default rule configuration: requests that do not match any existing routing rule are routed to the Workspace corresponding to the default rule (#30)
The new version removes the ability to configure Sinker directly in host installation mode. This feature will be supported in a new way in a future release.
Removed redundant /v1/write/xxx APIs and added the /v1/write/:category endpoint
Write data is no longer decoded as line protocol, but the body is still read for signing
Disk cache:
When sending to the central platform fails during cache cleanup, a consumption pause policy was added so that the current cached request must be sent successfully before the next cache entry is cleaned