Continuing a platform's unsampled traceparent under a parent-based sampler drops every span of the request. Measured with the OpenTelemetry SDK; four header policies compared.
Spans that vanish after an upgrade
ADK Python v2.14.1, released on 12 November 2026, contains a single fix, and it is a useful one to understand even if you never deploy to the platform it mentions. The commit, "turn off FastAPI's native tracing on Agent Engine", explains the chain of events. FastAPI 0.142.0, released on 29 September 2026, added native OpenTelemetry support, and FastAPI's documentation says tracing is on by default. Its server span continues the inbound traceparent header. On Agent Engine, the commit says, that header is the platform's, not the caller's, and Cloud Run marks it sampled for at most about 0.1 requests per second. Under the default parentbased_always_on sampler, every span of an unsampled request was dropped, ADK's own spans included. The fix passes telemetry={"tracing": False} to FastAPI when the Agent Engine environment variable is set, so ADK's spans start a new trace again, or continue a separate platform header when it is present.
The general lesson is about trust boundaries in trace context. A traceparent header carries a decision, the sampled flag, made by whoever set it. If you continue it, you inherit that decision. When the header was set by your caller, that is exactly what distributed tracing is for. When it was set by a platform hop you do not control, you may be inheriting "do not record" for most of your traffic.
The mechanism in three facts
First, the W3C Trace Context header has four fields: version, trace id, parent span id and trace flags. The lowest bit of the flags is the sampled flag. -01 at the end means the caller recorded its span and suggests you do too; -00 means it did not.
Second, the OpenTelemetry SDK's default sampler is ParentBased with AlwaysOn for roots. The specification's configuration table lists parentbased_always_on as the default for OTEL_TRACES_SAMPLER. A ParentBased sampler with no parent samples by its root rule, here always; with a remote parent, it follows the parent's sampled flag. So a remote parent with flags 00 produces a non-recording span, and every child of that span is non-recording too.
Third, a framework that creates the server span before your code runs decides the parent for everything below it. If FastAPI's middleware attaches the extracted context first, ADK's later decision to ignore the header does not help, because ADK's spans become children of a span that is already non-recording. The commit says exactly this: ADK already ignored the plain traceparent on Agent Engine, but FastAPI attached it as the current context before ADK's middleware ran.
A measurement you can repeat
The companion code uses the real OpenTelemetry Python SDK, version 1.45.2, with an in-memory exporter and ParentBased(root=ALWAYS_ON). Each simulated request for the fictional Belgavi's Book Store agent, handling CASE-8040, creates three spans: the server span, a model call and a tool call. It compares four ways to treat the inbound header.
from opentelemetry.trace.propagation.tracecontext import TraceContextTextMapPropagator
from opentelemetry.context import Context
prop = TraceContextTextMapPropagator()
platform_ctx = prop.extract({"traceparent": headers["traceparent"]})
parent = platform_ctx # continue: inherit the platform's decision
parent = Context() # ignore: start a new root trace
# link: new root, plus Link(platform span context) on the server span
# dedicated: continue only a caller-owned header, if present
with tracer.start_as_current_span("POST /run", context=parent):
...The recorded results:
inbound header mode spans exported
platform, flags 00 continue 0 of 3
platform, flags 00 ignore 3 of 3
platform, flags 00 link 3 of 3 (1 span link)
platform, flags 00 + caller dedicated 3 of 3, caller's trace id kept
platform, flags 01 any mode 3 of 3
no header any mode 3 of 3Continuing an unsampled platform parent exports nothing. That is not a bug in the SDK; it is the sampler doing what it was configured to do. The bug is in the choice of parent.
Four policies for an inbound header
Continue is right when the header comes from a caller you trust and want to correlate with: one of your own services, or a client that participates in your tracing.
Ignore is right at a public edge, or behind a platform that injects its own context. You lose correlation with the hop before you, which you usually did not have anyway.
Link is the compromise the OpenTelemetry specification provides for exactly this case. Start a new root, so your own sampler decides, and add a span link to the inbound span context. A trace backend that supports links can still navigate from your trace to the platform's when both exist.
Dedicated header is what the ADK fix does on Agent Engine: continue Google-Agent-Engine-Traceparent when it is set, according to the commit message, and ignore the plain traceparent, which the commit describes as the platform's. This keeps end-to-end correlation without inheriting the platform's sampling.
Deciding which headers to trust
Trust here is not about secrecy: a traceparent carries no credentials. It is about who gets to make decisions for you. Anyone who can send you a header can choose your trace id and, under a parent-based sampler, whether you record the request at all. A sensible rule is to trust inbound trace context only from network locations you control, such as your own gateway or an internal service mesh, and to treat everything else as untrusted input that may be linked but not continued. Write the rule down next to the sampler configuration, because the two only make sense together. When a platform or framework gives you a separate header for the caller's context, as ADK does on Agent Engine, prefer that over guessing which hop set the plain one.
Why this hits agent services harder
An agent request is long and expensive: several model calls, tool calls, retries, sometimes subagents. Its trace is often the only practical way to see what happened, which call failed, and where the tokens went. Losing those spans silently for most requests means evaluations based on production traces quietly sample the wrong population, cost dashboards undercount, and incident investigation finds empty traces. The failure is also invisible in development, where requests usually arrive without a platform header and every span is recorded.
What stays deterministic
Choosing a parent is configuration, not code scattered across handlers. Decide it once per entry point: which headers are trusted, from which network locations, and which policy applies. The model and the tools never touch trace context directly; they run inside spans the entry point created.
Migration steps
- List every entry point and what sits in front of it: load balancer, platform, gateway, mesh. Find out which of them inject or rewrite
traceparent. - Check whether your web framework creates server spans itself, and in which version that started. FastAPI does so from 0.142.0 with tracing on by default, and documents
telemetry={"tracing": False}to turn it off. - Make one component responsible for the server span, and set its parent policy explicitly.
- Prefer a link over continuation when the inbound context is not your caller's.
- Keep your own sampling decision at the root you control, and record it as an attribute so you can audit the rate later.
Failure modes
- Spans disappear after a dependency upgrade that added instrumentation.
- Two layers both create server spans, and the outer one decides the parent.
- A trusted internal header is honoured from the public internet, letting any caller turn your tracing off for its requests by sending flags
00. - Head sampling at the platform was tuned for the platform's cost, not for your observability needs.
- Error logs exist without traces. FastAPI's documentation notes that error logs are recorded even when the trace is not sampled, which can make the gap look like a tracing outage rather than a sampling decision.
When this does not apply
If your service is called only by your own services, all propagating context consistently and sampling at the edge, continuing the inbound header is correct and the default sampler is a good choice. The problem is specific to a parent whose sampling decision you did not make.
Verify it
Run the companion probe with your own SDK version. In your service, send three requests in a test: no header, a header with flags 00 and a header with flags 01, and assert the number of exported spans for each with an in-memory exporter. Add the test to the suite that runs when you upgrade the web framework or the platform image.
Run the example yourself
The four inbound-header policies measured with the real OpenTelemetry Python SDK 1.45.2 and an in-memory exporter, with five tests. No web framework or collector is involved. Companion code on GitHub; its README lists the prerequisites, exact commands, expected output and limits.