Skip to content

feat(storage): speculation path set store - #501

Open
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store
Open

feat(storage): speculation path set store#501
behinddwalls wants to merge 1 commit into
preetam/speculation-standardfrom
preetam/speculation-path-store

Conversation

@behinddwalls

@behinddwalls behinddwalls commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

Why?

The speculate run reads every head's path set at the start of a run and persists what it funds at the end. entity.SpeculationPathSet exists, but nothing can store one.

What?

Adds storage.SpeculationPathSetStore, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.

The mutation is a conditional whole-item put — Update replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own Version field is ignored, per the storage README.

No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.

Test Plan

bazel test //submitqueue/extension/storage/... — sqlmock coverage of get/create/update and every error contract, plus a pin that Update ignores the entity's Version field.

bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.

make fmt, make gazelle, make mocks

Issues

Stack

  1. feat(speculation): generator contract and bestfirst impl #446
  2. feat(speculation): allocator contract and sticky impl #450
  3. feat(speculation): standard composed speculator #451
  4. @ feat(storage): speculation path set store #501
  5. feat(storage): path-build link store for per-path builds #502
  6. feat(orchestrator): dispatch builds per speculation path #503
  7. feat(orchestrator): poll builds and stop the ones nothing wants #504
  8. fix(orchestrator): mint distinct message IDs for cancel re-publishes #505
  9. feat(orchestrator): re-plan the queue from the Speculator each run #506
  10. feat(orchestrator)!: finalize batches from their speculation paths #507
  11. feat(orchestrator): compose per-queue speculators and turn speculation on #508

## Summary

### Why?

The speculate run reads every head's path set at the start of a run and persists what it funds at the end. `entity.SpeculationPathSet` exists, but nothing can store one.

### What?

Adds `storage.SpeculationPathSetStore`, keyed on the head batch ID, with a MySQL implementation, schema, mocks, and contract tests.

The mutation is a conditional whole-item put — `Update` replaces the set, guarded on version — because a conditional put on a key is the primitive every backend offers directly; a field-level update is the one shape non-SQL backends would have to emulate with a read-modify-write. Version arguments are explicit and the entity's own `Version` field is ignored, per the storage README.

No secondary index: callers that want a queue's live sets enumerate the heads from the batch listing they already hold and read each set by key.

## Test Plan

✅ `bazel test //submitqueue/extension/storage/...` — sqlmock coverage of get/create/update and every error contract, plus a pin that `Update` ignores the entity's `Version` field.

✅ `bazel test //test/integration/submitqueue/extension/storage/mysql:go_default_test` — against real MySQL: round-trip (each stored ID still hashes its stored path), missing head, duplicate create, and a compare-and-swap race where the loser must not restore the path the winner dropped.

✅ `make fmt`, `make gazelle`, `make mocks`
@behinddwalls
behinddwalls force-pushed the preetam/speculation-path-store branch from e481adf to c4f175e Compare August 4, 2026 05:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant