Skip to content

docs: make multi-worker consistency semantics executable - #280

Open
dgenio wants to merge 5 commits into
mainfrom
agent/document-multi-worker-consistency
Open

docs: make multi-worker consistency semantics executable#280
dgenio wants to merge 5 commits into
mainfrom
agent/document-multi-worker-consistency

Conversation

@dgenio

@dgenio dgenio commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Closes #226.

What changed

  • Adds docs/deployment-consistency.md with an explicit component-by-component consistency matrix for multi-worker deployments.
  • Adds tests/test_multi_worker_consistency.py so the load-bearing claims are executable documentation rather than prose that can silently drift.

The tests pin four current facts:

  1. a signed token issued in worker A verifies in worker B when both share the signing secret;
  2. default in-memory revocation in A does not propagate to B;
  3. rate-limit windows are independent between workers;
  4. default handles are process-local and cannot be expanded/fetched from another worker.

Why

The mixed stateless/stateful model is non-obvious: sharing WEAVER_KERNEL_SECRET shares token verification ability, not revocation/limit/handle state. This is a security and operations boundary that adopters need to know before horizontal scaling.

Architecture decision

This PR deliberately does not build a sidecar or shared-state control plane. It documents the evidence first and states that #227 should earn its complexity from real adopter consistency requirements. A small shared-store protocol may be sufficient for some guarantees.

Validation

The new test file uses existing component APIs and should run in the normal CI matrix. No runtime behavior is changed.

@dgenio
dgenio marked this pull request as ready for review August 11, 2026 06:10
Copilot AI lite review requested due to automatic review settings August 11, 2026 06:11

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Makes the multi-worker consistency semantics explicit and verifiable by turning key deployment claims into executable tests and documenting a component-by-component consistency matrix.

Changes:

  • Added a new pytest module that asserts cross-worker behavior for token verification, revocation, rate limits, and handles.
  • Added a deployment consistency document that summarizes default state locality and links to the tests as evidence.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 6 comments.

File Description
tests/test_multi_worker_consistency.py Adds tests intended to “pin” multi-worker consistency semantics across key components.
docs/deployment-consistency.md Documents default consistency/state-sharing expectations and points to the new tests as executable evidence.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +13 to +15
def test_token_signature_verifies_across_workers_that_share_a_secret() -> None:
worker_a = HMACTokenProvider(secret=_SECRET)
worker_b = HMACTokenProvider(secret=_SECRET)
Comment on lines +25 to +27
def test_in_memory_revocation_does_not_propagate_between_workers() -> None:
worker_a = HMACTokenProvider(secret=_SECRET)
worker_b = HMACTokenProvider(secret=_SECRET)
)


def test_rate_limit_windows_are_process_local() -> None:
Comment on lines +52 to +53
worker_a = RateLimiter(clock=fixed_clock)
worker_b = RateLimiter(clock=fixed_clock)
Comment on lines +64 to +66
def test_default_handle_store_is_not_portable_between_workers() -> None:
worker_a = HandleStore()
worker_b = HandleStore()
Comment on lines +11 to +18
| Component | Default state | Two workers sharing the same secret | Consequence |
| --- | --- | --- | --- |
| capability-token signature verification | stateless HMAC | token issued by A verifies in B | expected and useful |
| token revocation | in-memory store unless replaced | revoking in A does not revoke in B | revocation is not globally consistent by default |
| rate-limit windows | in-memory | each worker has an independent window | an effective deployment-wide limit can scale with worker count |
| handles / expanded results | in-memory `HandleStore` | handle created in A is unknown in B | requests that move workers cannot expand that handle |
| in-memory traces | process-local | each worker sees its own trace store | audit history fragments unless a shared/durable store is used |
| budget/runtime counters | process-local where backed by in-memory state | counters can diverge | deployment-wide budgets require a shared coordination model |
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Investigate multi-process and multi-worker deployment semantics of in-memory kernel state

2 participants