We must implement open-feature/flagd#2027, which proposes changing the spec defaults so the stream-reconnect backoff can't leave us disconnected longer than the stale grace period.
config.py:31 DEFAULT_RETRY_BACKOFF_MAX: 12000 -> 5000
config.py:32 DEFAULT_RETRY_GRACE_PERIOD_SECONDS: 5 -> 10
Both resolvers also set maxAttempts to 3 (resolvers/grpc.py:105, resolvers/process/connector/grpc_watcher.py:82), but the spec's retry policy specifies 4 (the initial attempt plus retries at 1s, 2s, 4s); should be 3 -> 4. Harmless to fix now since nothing clamps at a 5000 cap.
Separately, we don't emit STALE on sync-stream errors. _state_change_callback (grpc_watcher.py:178) only emits STALE when the gRPC channel enters TRANSIENT_FAILURE, and a stream-level error doesn't change channel state, so _handle_rpc_error (:273) logs at debug and we silently wait retry_backoff_max_ms in _wait_before_reconnect (:318) before re-establishing. For that whole window we're disconnected while still reporting READY.
Measured on 0.5.2, in-process resolver, flagd latest:
| Scenario |
Result |
stream_deadline_ms=3000, flagd untouched |
stream dies at 3.01s, reconnects at 15.01s, zero lifecycle events |
| default deadline, flagd killed for 1.3s |
zero lifecycle events |
With the default stream_deadline_ms of 600000 that's a silent 12s disconnect every ~10 minutes. Java, Go and JS all emit STALE on stream error here, and per the spec a stream disconnect should emit STALE, then ERROR after retryGracePeriod.
We must implement open-feature/flagd#2027, which proposes changing the spec defaults so the stream-reconnect backoff can't leave us disconnected longer than the stale grace period.
config.py:31DEFAULT_RETRY_BACKOFF_MAX: 12000 -> 5000config.py:32DEFAULT_RETRY_GRACE_PERIOD_SECONDS: 5 -> 10Both resolvers also set
maxAttemptsto 3 (resolvers/grpc.py:105,resolvers/process/connector/grpc_watcher.py:82), but the spec's retry policy specifies 4 (the initial attempt plus retries at 1s, 2s, 4s); should be 3 -> 4. Harmless to fix now since nothing clamps at a 5000 cap.Separately, we don't emit STALE on sync-stream errors.
_state_change_callback(grpc_watcher.py:178) only emits STALE when the gRPC channel entersTRANSIENT_FAILURE, and a stream-level error doesn't change channel state, so_handle_rpc_error(:273) logs at debug and we silently waitretry_backoff_max_msin_wait_before_reconnect(:318) before re-establishing. For that whole window we're disconnected while still reporting READY.Measured on 0.5.2, in-process resolver, flagd
latest:stream_deadline_ms=3000, flagd untouchedWith the default
stream_deadline_msof 600000 that's a silent 12s disconnect every ~10 minutes. Java, Go and JS all emit STALE on stream error here, and per the spec a stream disconnect should emit STALE, then ERROR afterretryGracePeriod.