Record the doctest core architecture as an ADR

docs/adrs/0002-runner-conformance-across-cpython.md

@@ -0,0 +1,112 @@
+(adr-0002-runner-conformance-across-cpython)=
+
+# ADR 0002: Runner conformance across CPython versions
+
+Status: Draft
+Date: 2026-08-02
+
+## Context
+
+{doc}`0001-typed-vanilla-doctest-core` decides that the runner owns the
+per-example loop by defining `_DocTestRunner__run` in a subclass, rather than
+cloning CPython's code object or rebinding `doctest.compile` process-wide.
+
+Owning the loop means owning the private state it writes into, and that state has
+changed shape inside this project's supported interpreter range. Three
+divergences are known:
+
+**The outcome accumulator changed name and arity.** On 3.10 through 3.12 it is
+`__record_outcome(self, test, f, t)` writing into `self._name2ft`; on 3.13 and
+later it is `__record_outcome(self, test, failures, tries, skips)` writing into
+`self._stats`
+([`Lib/doctest.py:1485`](https://github.com/python/cpython/blob/v3.14.2/Lib/doctest.py#L1485)).
+A loop that calls the wrong one leaves `summarize()` reporting zeros for a
+passing file — a silent, total failure of the reporting path.
+
+**`TestResults` gained a third value that is not a tuple field.** It carries
+`skipped` as an extra instance attribute
+([`Lib/doctest.py:114`](https://github.com/python/cpython/blob/v3.14.2/Lib/doctest.py#L114)),
+so `TestResults(f, a, skipped=s)` works on 3.13+ and raises on earlier versions.
+
+**`report_skip` does not exist at v3.14.2.** The runner has only `report_start`,
+`report_success`, `report_failure` and `report_unexpected_exception`
+([`Lib/doctest.py:1286-1314`](https://github.com/python/cpython/blob/v3.14.2/Lib/doctest.py#L1286-L1314)).
+It appears in later prereleases, so a loop must probe rather than assume in
+either direction.
+
+A fourth risk has no current instance but would be silent: a CPython refactor
+that inlines the loop into `run()` would route execution back to stdlib. That is
+invisible for prompt-form blocks and immediately broken for `{testcode}`.
+
+## Question
+
+How is an owned per-example loop proven equivalent to the interpreter's own,
+continuously, without a `sys.version_info` ladder?
+
+## Direction
+
+A differential conformance harness, run in CI on every supported interpreter,
+gating the build step that lands the runner.
+
+**Scoped to the extended lane.** {doc}`0001-typed-vanilla-doctest-core` runs
+ordinary prompt blocks on CPython's untouched per-example loop, so those need no
+differential proof — they *are* the reference. The owned `__run` is invoked only
+for `exec` bodies, top-level await and future profiles, and that is what this
+harness guards. It is a smaller obligation than an unconditionally owned loop,
+and it is the reason owning the loop is affordable at all.
+
+A fixed case matrix — pass, fail, unexpected exception, `SyntaxError`, all
+examples skipped, partially skipped, `FAIL_FAST`, `REPORT_ONLY_FIRST_FAILURE`,
+`IGNORE_EXCEPTION_DETAIL`, and an exec-mode body — is run through both this
+runner and a stock {class}`doctest.DocTestRunner`, asserting the captured
+`report_*` text, `summarize()` output, the accumulator contents, and the result
+as `(failed, attempted, skipped)`.
+
+**Assert the triple, not `TestResults` equality.** `TestResults` is a two-field
+namedtuple carrying `skipped` off-tuple, so `==` compares only two of the three
+values and a skip-count regression passes silently. `attempted` is also
+incremented *before* the `SKIP` check, so a skip that wrongly executes moves
+neither counter — it is invisible to both the tuple and to `summarize()` at zero
+failures, and only the `report_*` text distinguishes it.
+
+The exec-mode case is the one the two runners are *meant* to disagree on, and it
+still compares against stock. `compile()` raises on a multi-statement body, but
+that call sits inside the loop's own `try`
+([`Lib/doctest.py:1398-1408`](https://github.com/python/cpython/blob/v3.14.2/Lib/doctest.py#L1398-L1408)),
+so a stock {class}`doctest.DocTestRunner` catches the `SyntaxError` and records
+it as an unexpected exception rather than propagating it: one
+`report_unexpected_exception` call, `TestResults(failed=1, attempted=1)`, and
+`_stats` at `(1, 1, 0)`. Only {class}`doctest.DebugRunner` — and pytest's runner
+beneath it — converts that into a raise, as
+{exc}`doctest.UnexpectedException`. So the case is asserted as a pair: stock
+records the failure, this runner records a pass. A regression that silently
+reverts to `"single"` mode shows up as the two converging.
+
+**What else belongs in the matrix, and what does not.** Add `report_*` hook
+events — the only channel that distinguishes a skip which wrongly *executed*,
+since `attempted` increments before the `SKIP` check and neither counter moves —
+and repeated runs of one test, which exercise accumulator arithmetic across
+calls.
+
+Cross-block `FAIL_FAST` and cleanup aggregation stay out. Both are properties of
+`run_group()` rather than of the per-example loop, so a stock runner offers
+nothing to compare them against; they belong to
+{doc}`0001-typed-vanilla-doctest-core`'s item-lifecycle tests. A
+{exc}`pytest.skip` raised inside an example and a debugger exit are likewise
+pytest-layer concerns, testable only through a pytest session.
+
+Version handling is by capability probe, never by version comparison, so a
+backport, a vendored interpreter or a fork behaves correctly rather than by
+coincidence. {doc}`0001-typed-vanilla-doctest-core` rejects an import-time guard
+that raises: a `pytest11` plugin that aborts at import takes down suites whose
+majority of tests never touch a doctest.
+
+## Open
+
+- Whether the harness asserts on `report_*` text verbatim, or on a normalized
+  form — verbatim is stricter and will churn when CPython adjusts wording.
+- Whether a probe failure degrades to stdlib's loop with a diagnostic, or fails
+  the affected items loudly. Degrading is silent for prompt-form blocks, which is
+  the argument against it.
+- The floor: whether supporting 3.10's `_name2ft` shape is worth its shim once
+  that version reaches end of life.

docs/adrs/0003-rejecting-per-block-items.md

@@ -0,0 +1,79 @@
+(adr-0003-rejecting-per-block-items)=
+
+# ADR 0003: Rejecting per-block items over a shared mapping
+
+Status: Draft
+Date: 2026-08-02
+
+## Context
+
+[PR #87](https://github.com/git-pull/gp-libs/pull/87) proposes two settings that
+together choose how a page's blocks are collected. One of them,
+`doctest_docutils_namespace_items = per-block`, keeps a node id for every block
+of a shared page and hands those blocks one live `globs` mapping rather than
+merging them into a single test.
+
+**Neither setting has shipped.** Both live on an open branch, in no release and
+on no tag. There is nothing to deprecate, and this record does not propose a
+deprecation — it records why the shape should not ship.
+
+## The shape, and why it is attractive
+
+`per-block` answers a real complaint about merging. Merging a group into one
+`DocTest` collapses N node ids into one, merges fixture lifetime across the whole
+group, and makes the failure gutter span the page. Keeping one id per block fixes
+all three, and on a large documentation tree the difference is the bulk of the
+suite's visible granularity.
+
+## Why it should not ship
+
+**A node id that cannot be selected is not a node id.** Selecting block three of
+a stateful page raises `NameError`, because the blocks that bound the names it
+reads did not run. The id promises an addressable unit and does not deliver one.
+
+**A live mapping cannot cross a process.** Only execnet-serializable builtins
+reach an xdist worker, so the shape needs a scheduler that keeps a page whole —
+and the only affinity primitive in xdist is
+[`_split_scope(nodeid) -> str`](https://github.com/pytest-dev/pytest-xdist/blob/v3.8.0/src/xdist/scheduler/loadscope.py#L284).
+Under a user-typed `--dist load` there is no scheduler to influence, so the
+options collapse to refusing the run.
+
+**A live mapping cannot survive an item running twice.** A retry re-runs a block
+against globals it already mutated, so an expectation true only on the second
+attempt reports as a pass. Guarding that means refusing reruns.
+
+**A worker crash re-runs only the uncompleted tail** of a work unit, on a fresh
+process — so blocks 3..N of a shared group run against an empty mapping, and
+worker restarts are on by default. This one has no guard at all.
+
+Those four are why the branch also carries a worker-count fork, a page-inference
+heuristic over node-id strings, a scheduler substitution, a scheduler refusal and
+a rerun refusal. The guards are the cost of the shape, not incidental.
+
+## Decision
+
+Do not ship per-block items over a shared mapping, under this or any spelling.
+
+{doc}`0001-typed-vanilla-doctest-core` reaches the same granularity goal from the
+other side: one item per group, holding one `DocTest` per block. That gives
+per-block failure locations, gutters and "location unknown" without a shared
+mapping ever becoming schedulable, so none of the four guards is needed.
+
+What it does not give is a per-block *outcome* or a per-block *node id*. That
+limit is honest and is recorded in {doc}`0001-typed-vanilla-doctest-core`'s
+outcome contract, rather than papered over with an id that raises when used.
+
+## Consequences
+
+Because nothing shipped, there is no migration path to write, no deprecation
+warning to add and no downstream grep to run.
+
+## Open
+
+- Whether a human-facing block *label* — in failure text and the report header,
+  never as a node id — is worth adding later, so a reader can find the failing
+  block without the design promising `-k` isolation. Not in a first version.
+- Whether `--doctest-docutils-namespace-scope` should be renamed to
+  `--doctest-docutils-share` before or after this architecture lands.
+  {doc}`0001-typed-vanilla-doctest-core` settles the vocabulary; the rename is
+  independently schedulable and, since neither spelling has shipped, cheap.

docs/adrs/0004-diagnostics-as-data.md

@@ -0,0 +1,105 @@
+(adr-0004-diagnostics-as-data)=
+
+# ADR 0004: Diagnostics as data
+
+Status: Draft
+Date: 2026-08-02
+
+## Context
+
+Parsing a page currently writes docutils reporter output straight to stderr,
+interleaved with pytest's own output and attributable to nothing. Two failure
+modes follow from the default settings.
+
+A level-4 message raises `SystemMessage` mid-parse and aborts collection of the
+file, so one malformed construct takes down a page whose other blocks are fine.
+
+More quietly, a `.. doctest::` block carrying an unknown option collects **zero**
+tests and the session exits green. A page that checks nothing reports the same
+way as a page that passes.
+
+{doc}`0001-typed-vanilla-doctest-core` gives the front-end layer a second return
+value for this: `Diagnostic(level, code, message, path, line)`.
+
+**Two mechanism assumptions in the first draft were wrong, and the fix is not
+cosmetic.**
+
+*There is no stable code to key on.* A docutils `system_message` carries a level
+and text, and nothing semantically stable. So codes exist only for diagnostics
+**this project emits**; docutils-originated messages arrive code-less and have to
+be *classified* before they can be suppressed or promoted. The classifier is the
+open question below, and it cannot be "key on the code", because for these
+messages there is none.
+
+*An observer does not silence the stream.* Attaching one is additive: the message
+still reaches the warning stream. Turning reporter output into values needs three
+settings together — `halt_level` above 4 (both to avoid the mid-parse abort and
+because a halting message bypasses observer notification entirely),
+`report_level` at 5 or `warning_stream` disabled to stop the write, and then the
+observer.
+
+## Question
+
+Which diagnostics are shown by default?
+
+The naive answer — show everything — was measured against this project's own
+`docs/` and produces well over a hundred messages per run, almost all of them
+`Unknown interpreted text role` and `Unknown directive type` for roles and
+directives that Sphinx supplies and a bare-docutils parse structurally cannot
+resolve. Those are false positives. Emitting them is noise-as-policy, and users
+would learn to ignore the channel that also carries real errors.
+
+The opposite error is worse: suppressing one code too many turns a broken page
+into a silent zero-test page, which is the exact condition this ADR exists to
+surface.
+
+## Direction
+
+**Treat unknown roles and unknown directives differently.** They are not the same
+risk, and the first draft's symmetric treatment was the mistake.
+
+An **unknown role** is inline markup. It cannot swallow a code block, so a
+bare-docutils parse seeing `:mod:` in a Sphinx project is noise and is suppressed
+by default.
+
+An **unknown body-owning directive** is a collection error. It swallows its body
+unparsed, so a page whose doctests live inside one collects zero tests and exits
+green — which is the failure diagnostics-as-data exists to prevent. Suppressing it
+by default trades a loud, correct error for a silent wrong answer. A project with
+legitimate foreign containers registers them as known vocabulary; that is an
+explicit act, not a default.
+
+For a Sphinx project the question does not arise: the extractor consumes an
+already-resolved doctree, in which every registered directive has run.
+
+"By code" remains the intent for everything else; the classifier that assigns a
+code to a docutils message is unsettled, which is why this record stays `Draft`.
+
+Every diagnostic raised by this project's own layers defaults to visible, and
+`level="error"` from those layers fails collection with the file and line named.
+A page whose only block fails to parse, and a page with a malformed `:options:`
+value, must both produce a collection error rather than collecting nothing and
+passing.
+
+Expose promotion and suppression by code so a project can tune the set without a
+global on/off switch.
+
+## Open
+
+- Whether diagnostics surface as {class}`pytest.PytestWarning` subclasses, giving
+  `-W error::` control for free, or as a dedicated report section.
+- **What classifies a code-less docutils message.** The options are an owned,
+  version-pinned message-text table with a test that fails on upstream rewording
+  (and which must handle two dialects — reST's `Unknown directive type "x".` at
+  ERROR/3 versus MyST's `Unknown directive type: 'x'` at WARNING/2), or
+  pre-empting at the source by overriding the directive-dispatch path so the
+  unknown case never becomes a reporter message at all. This is the decision
+  ADR 0004 cannot ship without.
+- **How a project registers a legitimate foreign container**, so that an unknown
+  body-owning directive it genuinely does not care about stops erroring. This is
+  the escape hatch the default requires, and it needs a spelling.
+- Whether a near-miss to a registered name (`.. doctset::` for `.. doctest::`)
+  earns a distinct, more helpful message than the generic unknown-directive
+  error. Cheap, and the typo is the common case.
+- Whether the CLI (`python -m doctest_docutils`) and the pytest plugin share one
+  formatter or two.