Bump conformance harness to 0.2.0-alpha.11

[maxisbey]

Bumps the pinned conformance harness from 0.2.0-alpha.10 to 0.2.0-alpha.11 and adds the one client fixture handler the new release needs.

Motivation and Context

alpha.11 is the harness the current SDK tier round is scored with. Relative to alpha.10 it adds JSON-schema validation of every wire message (conformance#399), a server-session-lifecycle scenario (#322), a best-effort DELETE after each stateful server scenario (#316), a frozen --requirements <revision> mode (#447), and one new client scenario, json-schema-2020-12-preservation (#335).

Only the last one needs a change here. The scenario lists tools and expects the client to echo the focal tool's inputSchema back through json_schema_echo, so the harness can detect 2020-12 keywords ($schema, $defs, $anchor, additionalProperties, allOf/anyOf, if/then/else) being stripped in transit. client.py had no handler, so both client legs failed with "Unknown scenario". With the handler it passes 9/9 (including the new wire-schema-valid check) at the default, 2025-11-25 and 2026-07-28 wires — Tool.input_schema is a plain dict, nothing is dropped.

No baseline entries change. The nine tasks-* server entries and the nine per-check DPoP/WIF client entries fail check-for-check as before; tasks-status-notifications still emits no checks and stays out.

Also fixes two bits of docstring drift in client.py (the exit budget is the harness --timeout, and http-custom-headers was missing from the scenario list).

How Has This Been Tested?

All six CI legs run locally through run-server.sh / run-client.sh against a build of conformance c321dd3 (its dist/index.js is byte-identical to the published alpha.11 tarball):

Leg	Result
server --suite active	73 passed / 0 failed (new server-session-lifecycle 3/3; 31 teardown DELETEs, all 200)
server --suite draft	103 / 0
server --suite all --spec-version 2026-07-28	151 / 0
server --suite all	204 / 25 — the nine tasks-*, all expected, none stale
client --suite all	458 / 9 — the nine per-check DPoP/WIF entries, all expected; no solo re-run needed
client --suite all --spec-version 2026-07-28	381 / 0

Reviewers will see one extra wire-schema-valid row per instrumented scenario and ~31 DELETE /mcp 200 lines per stateful server leg in the logs; both are expected.

For reference, main also scores 30/30 + 37/37 (server) and 18/18 + 32/32 (client) on the new --requirements 2025-11-25 / --requirements 2026-07-28 sets. Wiring those in as CI legs is a follow-up.

Breaking Changes

None. CI-only.

Types of changes

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to change)
• Documentation update

Checklist

• I have read the MCP Documentation
• My code follows the repository's style guidelines
• New and existing tests pass locally
• I have added appropriate error handling
• I have added or updated documentation as needed

Additional context

_{AI Disclaimer}

[claude]

LGTM — CI-only harness pin bump plus one new scenario handler that follows this file's existing patterns; the inline note is a failure-path diagnostics nit, not a correctness issue.

Extended reasoning...

Overview

This PR bumps the pinned conformance harness from 0.2.0-alpha.10 to 0.2.0-alpha.11 in .github/workflows/conformance.yml (a one-line version-string change) and adds a handler for the one new client scenario, json-schema-2020-12-preservation, to .github/actions/conformance/client.py, plus two docstring corrections (timeout wording, missing http-custom-headers entry in the scenario list). No SDK source, tests, or baseline files are touched.

Security risks

None. The change is confined to CI tooling: the new handler only lists tools against the harness mock server and echoes a schema dict back through a tool call. The workflow's pinned action SHAs and permissions are unchanged, and the harness package remains pinned to an exact version passed to npx --yes.

Level of scrutiny

Low-to-moderate. This is CI-only code with no production impact; the worst failure mode is a red conformance leg. I verified the new handler against the SDK surface it uses — Client(server_url, mode=...), list_tools(), call_tool(), and tool.input_schema all exist and are used identically by the neighboring handlers (run_http_invalid_tool_headers, run_tools_call). The handler structure, client_mode() usage, and docstring style all match the file's established conventions.

Other factors

The bug hunting system's one finding is a nit about the bare next() producing an opaque RuntimeError: coroutine raised StopIteration if the fixture tool is ever absent — a real but minor diagnostics-quality issue that only manifests on a future harness rename or SDK filtering change, and even then the leg fails either way. It does not affect the passing path and doesn't warrant blocking. The PR description reports all six CI legs run locally against a byte-identical build of alpha.11 with no baseline changes, and CI itself will exercise the same legs. Given the small, self-contained, CI-only scope, this is safe to approve.

[claude]

🟡 The bare next() on line 268 has no default, so if json_schema_2020_12_tool is ever absent from the tools/list result (a future harness pin bump renaming the fixture, or the SDK's x-mcp-header filtering dropping it), the escaping StopIteration surfaces as an opaque RuntimeError: coroutine raised StopIteration that names neither the missing tool nor the listing. Consider next((t for t in listed.tools if t.name == "json_schema_2020_12_tool"), None) followed by an explicit RuntimeError listing the surfaced tool names, matching this file's existing precondition-guard convention.

Extended reasoning...

What the issue is. run_json_schema_2020_12_preservation locates the focal tool with a bare next(tool for tool in listed.tools if tool.name == "json_schema_2020_12_tool") — no default argument. When the generator is exhausted without a match, next() raises StopIteration. Because this happens inside a coroutine, PEP 479 semantics kick in: a StopIteration escaping a coroutine frame is converted by the interpreter into RuntimeError: coroutine raised StopIteration. The resulting traceback names neither the missing fixture tool nor the tools/list result — it reads as interpreter internals, not as "the harness fixture wasn't found".

The code path that triggers it. The scenario handler lists tools via client.list_tools() and then searches for the harness-owned fixture tool by name. Two realistic triggers can make that search come up empty: (1) a future harness pin bump renames json_schema_2020_12_tool — this file exists precisely to track the pinned harness, and the workflow comment (conformance.yml) instructs bumping deliberately and reconciling baselines, so pin bumps are this file's expected lifecycle; (2) the SDK's own tool filtering drops the tool from the listing — run_http_invalid_tool_headers in this same file documents that the SDK filters tools with malformed x-mcp-header annotations out of list_tools results, so a filtering change in _absorb_tool_listing (src/mcp/client/session.py) is a second real path to an empty match.

Why existing code doesn't prevent it. Nothing guards the lookup. The rest of the file follows an explicit-diagnostics convention for missing preconditions — there are multiple raise RuntimeError("MCP_CONFORMANCE_CONTEXT missing 'client_id'")-style guards — but this handler skips that pattern for the one precondition it has.

Impact. CI-only. On the happy path (harness alpha.11 as pinned, where the PR reports 9/9 passing) nothing goes wrong. In the failure scenario the leg goes red either way; only the diagnostic quality differs — whoever does the next pin bump has to reverse-engineer coroutine raised StopIteration from a solo-rerun log instead of reading a message that points at the fixture and shows which tools were surfaced (which would also distinguish 'harness renamed the fixture' from 'SDK filtered the tool').

Fix. Identical passing-path behavior, strictly better failure diagnostics:

focal = next((t for t in listed.tools if t.name == "json_schema_2020_12_tool"), None)
if focal is None:
    raise RuntimeError(
        f"json_schema_2020_12_tool not in tools/list result: {[t.name for t in listed.tools]}"
    )

Step-by-step proof. Suppose harness alpha.12 renames the fixture tool to json_schema_2020_12_focal. (1) CI bumps CONFORMANCE_PKG; the scenario runs and listed.tools contains json_schema_2020_12_focal and json_schema_echo but no json_schema_2020_12_tool. (2) The generator inside next() is exhausted with no match, so next() raises StopIteration. (3) The exception propagates out of the coroutine frame run_json_schema_2020_12_preservation; per PEP 479, the interpreter replaces it with RuntimeError: coroutine raised StopIteration. (4) asyncio.run(handler(server_url)) in main() propagates that, the client process exits non-zero, and the harness records a failure whose only Python-side evidence is the interpreter-internals message — no mention of the tool name or the listing. With the fix, step (3) instead produces RuntimeError: json_schema_2020_12_tool not in tools/list result: ['json_schema_2020_12_focal', 'json_schema_echo'], immediately identifying the rename.

[maxisbey]

doesn't seem like an issue for a conformance test?

[cubic-dev-ai]

No issues found across 2 files

_{Re-trigger cubic}