[otel] Scope deterministic trace IDs without replacing provider-wide generation

[zhongkechen]

Summary

The OTel plugins install DeterministicIdGenerator directly on the resolved SDK TracerProvider, including global ADOT and caller-supplied providers. The generator stores the current execution trace ID in a shared mutable field and never clears it.

Consequently, any unrelated root span created by another instrumentation scope using the same provider can receive the durable execution trace ID.

Evidence

• packages/aws-durable-execution-sdk-python-otel/src/aws_durable_execution_sdk_python_otel/deterministic_id_generator.py::install_on_provider replaces provider.id_generator
• _execution_trace_id is provider-wide state set by set_trace_id() and used by every subsequent generate_trace_id() call
• Both plugins install the generator on global or explicit SDK providers during construction
• Neither invocation cleanup path clears or restores provider ID-generation state

A focused probe starts a durable execution, then starts an unrelated root span through provider.get_tracer("unrelated-library") with an empty context. The unrelated span receives the same trace ID as the workflow span.

OpenTelemetry requirements

The Trace API defines a trace as having a single root span and requires a new trace ID for a new root span:

https://opentelemetry.io/docs/specs/otel/trace/api/#span-creation

The SDK permits custom ID generators, but generated IDs must retain the required randomness and uniqueness characteristics:

https://opentelemetry.io/docs/specs/otel/trace/sdk/#id-generators

Impact

• Unrelated root spans can be merged into a durable execution trace.
• A trace can contain multiple unrelated root spans.
• Global ADOT or caller-owned provider behavior is changed for every instrumentation library, not only this plugin.
• Concurrent plugin instances or executions can overwrite one another’s current deterministic trace ID.
• Sampling and trace visualization can become misleading.

Acceptance criteria

• Durable deterministic ID selection is scoped to spans created by the durable OTel plugin.
• The plugin does not change trace-ID generation for unrelated instrumentation scopes.
• An unrelated root span receives a fresh trace ID before, during, and after a durable invocation.
• Concurrent plugin instances and executions cannot overwrite each other’s trace-ID state.
• Global ADOT, explicit-provider, and auto-provider modes are covered by tests.
• Stable cross-invocation workflow and operation correlation is preserved.