Trace sampling: what disappeared before the decision?

Every arrived trace

Compare policies on the same trace cohort

Forty uniquely numbered traces carry reproducible start times, durations and error outcomes. Head and tail panels operate on those same identities. Changing a policy preserves the generated outcomes; changing Seed creates a different cohort. Starts per second changes arrival timing without changing those trace durations and labels. Twenty percent error probability and twenty-five percent long-duration probability are synthetic fixture choices, not measured service statistics. Normal durations fall between one hundred and five hundred milliseconds; long durations between fifteen hundred and six thousand. The display reveals whether a trace was an error or slow only after it completes.

Head sampling chooses without the future outcome

Head keep percentage compares a fixed per-trace score with the chosen percentage at trace start. Zero keeps none and one hundred selects every trace unless its independent export cap has been reached. An accepted head trace is marked K immediately, but its contribution to interesting-completed recall is counted only after completion. No error or duration is consulted by that head decision. Export cap is a count for the entire forty-trace cohort, applied separately to both strategies. It is not a bytes-per-second budget or a measured storage bill. Earlier eligible selections can exhaust it before later interesting traces arrive.

Tail sampling needs admission and a bounded wait

Upstream admission percentage is a separate gate before the tail buffer. A trace rejected there cannot be recovered by a downstream policy. Tail buffer slots bounds how many traces can be waiting at once; a new arrival is dropped if all slots are occupied. Maximum wait starts at that trace arrival. This simplified model requires the complete trace to arrive before the deadline, then decides immediately on completion; an incomplete trace at the deadline is dropped. Completion or timeout frees its slot before a same-time arrival is admitted. This is an explicit teaching policy, not a reproduction of an OpenTelemetry Collector configuration or partial-span decision algorithm.

Choose what tail sampling considers useful

A completed error or a trace whose duration is at least Slow threshold qualifies for tail retention. Otherwise Normal keep percentage compares another fixed score with the chosen percentage. The policy still cannot exceed its export cap. The interesting-completed denominator includes all completed errors or threshold-slow traces in the original cohort, including those dropped upstream, for buffer pressure or timeout. If none has completed, recall is unknown rather than one hundred percent. The selected count and recall answer different questions: retaining many normal traces can consume capacity without increasing interesting-trace coverage.

Make resource limits defeat the idealized policy

Restore defaults and show the final result. Increase Tail buffer slots to forty, Maximum wait to eight seconds and Export cap to forty. Every interesting trace now fits and completes before its deadline, so tail interesting-completed recall reaches one hundred percent for this fixture. Reduce Maximum wait to one second and inspect timeout-drop rows: long traces cannot be retained by this complete-trace policy. Restore eight seconds, set Starts per second to five and reduce slots to one. Inspect buffer-drop rows and peak occupancy. Set Upstream admission percentage to zero to demonstrate that no tail policy can rescue absent traces. These experiments invalidate an unconditional promise that tail sampling always keeps all important traces.

Inspect and replay the comparison

Play maps the generated horizon onto sixty inspection seconds; the modeled seconds are shown separately. Pause holds the view, Restart rewinds, Next inspection second advances one viewing second and Show final result reaches the end. Speed changes viewing pace; Inspect time scrubs. Parameter edits rebuild and pause the comparison. Next seed changes only the seed; Restore defaults resets every control. The table gives each arrived trace a head decision and a specific tail state. K means selected, D dropped, B buffering and a dash not started. Exact evidence also exposes the full synthetic cohort for auditing, including future fixture outcomes; those future values are not inputs to the head decision. No distributed span routing, real trace IDs, collector protocol, late-span cache, byte accounting, adaptive controller or probability-weighted unbiased estimation is implemented.

Production reference, with different implementation details: OpenTelemetry Collector tail-sampling processor.