diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9f46fcd8..ecfbd399 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -51,6 +51,12 @@ jobs: - name: Typecheck (tsc --noEmit) run: npx nx run-many --target=typecheck --projects=app-web + # Informe, no puerta: hoy 14 de las 25 historias «usable» no tienen prueba que las nombre, + # así que bloquear haría nacer el gate en rojo — y un gate que nace en rojo se ignora. Sí + # bloquea si una spec se etiqueta con una historia que el tracker no declara usable (TD-012). + - name: Story coverage report (e2e ↔ functional stories) + run: npx nx run-many --target=cobertura-historias --projects=app-web + - name: Run Unit Tests # `--code-coverage` no es un flag válido de vitest/dotnet (vitest usa `--coverage`). # Se corre el target `test` (vitest run); la cobertura se gestiona con test:coverage. diff --git a/docs/architecture/technical-debt.md b/docs/architecture/technical-debt.md index 175e9d6c..6de37072 100644 --- a/docs/architecture/technical-debt.md +++ b/docs/architecture/technical-debt.md @@ -316,15 +316,36 @@ The write test that caused the 133 is **kept, deliberately, as a sentinel**: it --- +--- + ## [TD-012] Nothing Connects a Functional Story to the Test That Backs It -- **Status**: Confirmed +- **Status**: **Resolved — measurement in place** (2026-08-10). The gap it measures is real and open. - **Severity**: Medium -- **Component**: `docs/governance/project*/functional-story-gap-tracker.md`, `src/apps/ums.web-app/tests/*.spec.ts`, `docs/architecture/traceability-matrix.md` -- **Description**: The gap tracker declares **25 stories "Implemented / usable"**. Nothing verifies that claim. There are 23 e2e spec files and **84 cases**, and exactly **one** names a story — `FS-26`, which is not among the 25. The traceability matrix maps FS → ADR → Enabler, never FS → test. -- **How it surfaced (2026-08-10)**: a verification pass run on request. All 25 stories' REST surfaces answer against a live API, so the backend is not the issue. The issue is that the three defects fixed that same day were **front-end** defects, invisible from the API: `FS-20`/`FS-13` saved `false` for every boolean parameter regardless of the switch, and `FS-05` submitted `undefined` as the user id. Three stories sat green in the tracker while broken. -- **Measured coverage**: by keyword, `FS-16`, `FS-38` and `FS-39` have **no e2e trace at all**; `FS-09` and `FS-15` have one file each. The rest have de-facto coverage under feature-named specs (`profile-panel`, `role-state`, `parameter-catalog-state`…), but it is inferred from filenames, not declared. -- **Impact**: the column reads as verification and is intent. Anything hanging off it — a release scorecard, a date commitment, a conversation with product — inherits that gap. And a real regression can sit in a green row indefinitely, which is exactly what happened. -- **Suggested resolution**: name the story in each e2e case — `test('FS-05 · assign template to profile', …)` — and have CI assert that every story marked usable has at least one passing case. Same shape as the fix in [TD-004](#td-004): it adds no work, and it makes the claim impossible to hold while false. -- **Related**: the same disease as TD-002, TD-003, TD-010 and G-066 — a document asserting a state nobody checks against the code. This is its last known instance in this repository. +- **Component**: `docs/governance/project*/functional-story-gap-tracker.md`, `src/apps/ums.web-app/tests/*.spec.ts` +- **Description**: The tracker declared **25 stories "Implemented / usable"** and nothing verified it. 23 e2e spec files, 84 cases, and exactly one named a story — `FS-26`, not among the 25. The traceability matrix maps FS → ADR → Enabler, never FS → test. +- **How it surfaced**: three of those 25 were genuinely broken on 2026-08-10 — `FS-20`/`FS-13` saved `false` for every boolean parameter, `FS-05` submitted `undefined` as the user id — and all three sat green here. They were front-end defects, invisible from the API, so a REST sweep would never have found them. + +### Resolution + +Each spec declares the stories it answers to in its first line (`// @story FS-05, FS-24`), and `scripts/verificar-cobertura-historias.mjs` derives coverage **from the code**, comparing it against the tracker's own table. A separate mapping file was rejected: it would have drifted exactly like everything else this register records. + +`npm run cobertura:historias`, wired into CI ahead of the unit tests. + +**First measurement — 11 of 25 covered, 14 not:** + +| | Stories | +| :-- | :-- | +| **Covered** (11) | FS-01, FS-02, FS-03, FS-04, FS-05, FS-13, FS-14, FS-17, FS-20, FS-24, FS-40 | +| **No e2e naming them** (14) | FS-06, FS-07, FS-08, FS-09, FS-10, FS-11, FS-15, FS-16, FS-18, FS-19, FS-21, FS-22, FS-38, FS-39 | + +Three specs answer to no story — `authorization-ui`, `navigation`, `profile-panel` — legitimately: they test UI mechanics, not business outcomes. `profile-panel` is worth naming: the filename sounds like FS-05 and every case is avatar and drawer behaviour. Mapping by filename would have credited FS-05 with coverage it does not have, which is why this was done case by case. + +### Why it reports instead of blocking + +With 14 of 25 uncovered, a hard gate would be red from birth, and a gate that is red from birth gets ignored. It **does** fail on one condition: a spec tagged with a story the tracker does not declare usable — a labelling error, and a test claiming to cover something unrecognised is worse than no label. + +Promoting it to a gate (`--estricto`) is a decision for when the 14 are closed, not before. + +- **The underlying gap stays open**: 14 stories are declared usable with nothing exercising them end to end. What changed is that the number is measured on every CI run instead of assumed. diff --git a/src/apps/ums.web-app/package.json b/src/apps/ums.web-app/package.json index 1a93d982..79f21502 100644 --- a/src/apps/ums.web-app/package.json +++ b/src/apps/ums.web-app/package.json @@ -9,6 +9,7 @@ "lint": "eslint .", "lint:fix": "eslint . --fix", "typecheck": "tsc --noEmit -p tsconfig.app.json && tsc --noEmit -p tsconfig.node.json", + "cobertura:historias": "node scripts/verificar-cobertura-historias.mjs", "format": "prettier --write .", "format:check": "prettier --check .", "test": "vitest run", diff --git a/src/apps/ums.web-app/project.json b/src/apps/ums.web-app/project.json index 91bd4aba..faac0d57 100644 --- a/src/apps/ums.web-app/project.json +++ b/src/apps/ums.web-app/project.json @@ -20,6 +20,12 @@ "options": { "cwd": "apps/ums.web-app" } + }, + "cobertura-historias": { + "command": "npm run cobertura:historias", + "options": { + "cwd": "apps/ums.web-app" + } } } } diff --git a/src/apps/ums.web-app/scripts/verificar-cobertura-historias.mjs b/src/apps/ums.web-app/scripts/verificar-cobertura-historias.mjs new file mode 100644 index 00000000..3b43c511 --- /dev/null +++ b/src/apps/ums.web-app/scripts/verificar-cobertura-historias.mjs @@ -0,0 +1,78 @@ +#!/usr/bin/env node +/** + * verificar-cobertura-historias.mjs — ¿qué historia respalda cada prueba e2e? (TD-012) + * + * El tracker declara historias «Implemented / usable» y hasta ahora nada lo comprobaba: 84 casos + * e2e y uno solo nombraba una historia. Tres historias podían estar rotas —y lo estuvieron el + * 2026-08-10— sin que la fila dejara de estar en verde. + * + * La cobertura se deriva del CÓDIGO, no de un documento aparte: cada spec declara a qué historias + * responde con un `// @story FS-NN` en su primera línea. Un fichero de mapeo se habría + * desincronizado igual que se desincronizó todo lo demás. + * + * Salida: informe, no puerta. Hoy 14 de las 25 no tienen cobertura, así que bloquear sería nacer + * en rojo — y una puerta que nace en rojo se aprende a ignorar. Se bloquea cuando `--estricto`. + */ +import { readFileSync, readdirSync } from 'node:fs'; +import { join, dirname } from 'node:path'; +import { fileURLToPath } from 'node:url'; + +const AQUI = dirname(fileURLToPath(import.meta.url)); +const DIR_PRUEBAS = join(AQUI, '..', 'tests'); +const TRACKER = join(AQUI, '..', '..', '..', '..', 'docs', 'governance', 'project', 'functional-story-gap-tracker.md'); + +/** Historias que el tracker declara implementadas y usables. */ +function historiasDeclaradasUsables() { + const md = readFileSync(TRACKER, 'utf8'); + // La fila de la TABLA, no la prosa: el propio tracker lleva una nota que cita esa etiqueta + // entre comillas, y buscarla por texto suelto engancha el párrafo en vez de los datos. + const fila = md.split('\n').find(l => l.startsWith('|') && l.includes('Implemented / usable')); + if (!fila) throw new Error('No se encontró la fila «Implemented / usable» en la tabla del tracker.'); + return [...new Set(fila.match(/FS-\d+/g) ?? [])]; +} + +/** Historias que cada spec declara cubrir, leídas del propio fichero. */ +function coberturaDeclaradaEnPruebas() { + const porHistoria = new Map(); + const sinHistoria = []; + for (const f of readdirSync(DIR_PRUEBAS).filter(n => n.endsWith('.spec.ts'))) { + const primera = readFileSync(join(DIR_PRUEBAS, f), 'utf8').split('\n', 1)[0]; + const historias = primera.startsWith('// @story') ? (primera.match(/FS-\d+/g) ?? []) : []; + if (historias.length === 0) sinHistoria.push(f); + for (const h of historias) porHistoria.set(h, [...(porHistoria.get(h) ?? []), f]); + } + return { porHistoria, sinHistoria }; +} + +const usables = historiasDeclaradasUsables(); +const { porHistoria, sinHistoria } = coberturaDeclaradaEnPruebas(); + +const conCobertura = usables.filter(h => porHistoria.has(h)); +const sinCobertura = usables.filter(h => !porHistoria.has(h)); + +console.log(`\nHistorias declaradas usables: ${usables.length}`); +console.log(` con prueba e2e que las nombra: ${conCobertura.length}`); +console.log(` SIN ninguna: ${sinCobertura.length}\n`); + +for (const h of conCobertura) console.log(` ✓ ${h} ${porHistoria.get(h).join(', ')}`); +if (sinCobertura.length) { + console.log('\n Sin cobertura declarada:'); + for (const h of sinCobertura) console.log(` ✗ ${h}`); +} +if (sinHistoria.length) { + console.log(`\n Specs sin @story (no responden a ninguna historia): ${sinHistoria.join(', ')}`); +} + +// Una historia etiquetada que el tracker NO declara usable es un error de etiqueta, y ese sí +// bloquea siempre: significa que una prueba afirma cubrir algo que el tracker no reconoce. +const desconocidas = [...porHistoria.keys()].filter(h => !usables.includes(h)); +if (desconocidas.length) { + console.error(`\nERROR: specs etiquetadas con historias que el tracker no declara usables: ${desconocidas.join(', ')}`); + process.exit(1); +} + +if (process.argv.includes('--estricto') && sinCobertura.length) { + console.error(`\nERROR (--estricto): ${sinCobertura.length} historias usables sin prueba.`); + process.exit(1); +} +console.log(''); diff --git a/src/apps/ums.web-app/tests/app-configuration-state.spec.ts b/src/apps/ums.web-app/tests/app-configuration-state.spec.ts index 3ac3fdb2..ea76b102 100644 --- a/src/apps/ums.web-app/tests/app-configuration-state.spec.ts +++ b/src/apps/ums.web-app/tests/app-configuration-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-13 import { test, expect, type Page, type APIResponse } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/auth.spec.ts b/src/apps/ums.web-app/tests/auth.spec.ts index 5aa36190..877a6c76 100644 --- a/src/apps/ums.web-app/tests/auth.spec.ts +++ b/src/apps/ums.web-app/tests/auth.spec.ts @@ -1,3 +1,4 @@ +// @story FS-01 import { test, expect } from '@playwright/test'; test.describe('Authentication Flow', () => { diff --git a/src/apps/ums.web-app/tests/business-flows.spec.ts b/src/apps/ums.web-app/tests/business-flows.spec.ts index a3876076..841aa885 100644 --- a/src/apps/ums.web-app/tests/business-flows.spec.ts +++ b/src/apps/ums.web-app/tests/business-flows.spec.ts @@ -1,3 +1,4 @@ +// @story FS-02, FS-14 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/delegation-approval-gate.spec.ts b/src/apps/ums.web-app/tests/delegation-approval-gate.spec.ts index 449f771f..3c11b291 100644 --- a/src/apps/ums.web-app/tests/delegation-approval-gate.spec.ts +++ b/src/apps/ums.web-app/tests/delegation-approval-gate.spec.ts @@ -1,3 +1,4 @@ +// @story FS-14 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/feature-flag-state.spec.ts b/src/apps/ums.web-app/tests/feature-flag-state.spec.ts index 427f1d2d..71f4ff6a 100644 --- a/src/apps/ums.web-app/tests/feature-flag-state.spec.ts +++ b/src/apps/ums.web-app/tests/feature-flag-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-40 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/identity-provider.spec.ts b/src/apps/ums.web-app/tests/identity-provider.spec.ts index f4c63bc1..57590bb2 100644 --- a/src/apps/ums.web-app/tests/identity-provider.spec.ts +++ b/src/apps/ums.web-app/tests/identity-provider.spec.ts @@ -1,3 +1,4 @@ +// @story FS-03 import { test, expect, type Page, type APIResponse } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/login.spec.ts b/src/apps/ums.web-app/tests/login.spec.ts index 42e741d4..381f620c 100644 --- a/src/apps/ums.web-app/tests/login.spec.ts +++ b/src/apps/ums.web-app/tests/login.spec.ts @@ -1,3 +1,4 @@ +// @story FS-01 import { test, expect } from '@playwright/test'; test.describe('Login Flow', () => { diff --git a/src/apps/ums.web-app/tests/mvp-creation-chain.spec.ts b/src/apps/ums.web-app/tests/mvp-creation-chain.spec.ts index eca7279b..0a0efe6b 100644 --- a/src/apps/ums.web-app/tests/mvp-creation-chain.spec.ts +++ b/src/apps/ums.web-app/tests/mvp-creation-chain.spec.ts @@ -1,3 +1,4 @@ +// @story FS-02, FS-04 import { test, expect, Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/mvp-flows.spec.ts b/src/apps/ums.web-app/tests/mvp-flows.spec.ts index 0a31fbdc..396261bf 100644 --- a/src/apps/ums.web-app/tests/mvp-flows.spec.ts +++ b/src/apps/ums.web-app/tests/mvp-flows.spec.ts @@ -1,3 +1,4 @@ +// @story FS-03, FS-04 import { test, expect } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/parameter-catalog-state.spec.ts b/src/apps/ums.web-app/tests/parameter-catalog-state.spec.ts index 0a235778..17d5dcf4 100644 --- a/src/apps/ums.web-app/tests/parameter-catalog-state.spec.ts +++ b/src/apps/ums.web-app/tests/parameter-catalog-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-20 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/parameter-values.spec.ts b/src/apps/ums.web-app/tests/parameter-values.spec.ts index 787401e6..1f6df345 100644 --- a/src/apps/ums.web-app/tests/parameter-values.spec.ts +++ b/src/apps/ums.web-app/tests/parameter-values.spec.ts @@ -1,3 +1,4 @@ +// @story FS-13 import { test, expect, type Page, type APIResponse } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/permission-template-state.spec.ts b/src/apps/ums.web-app/tests/permission-template-state.spec.ts index 5cecc9d3..2a900890 100644 --- a/src/apps/ums.web-app/tests/permission-template-state.spec.ts +++ b/src/apps/ums.web-app/tests/permission-template-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-02 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/profile-slice.spec.ts b/src/apps/ums.web-app/tests/profile-slice.spec.ts index 4b7fda95..2b718382 100644 --- a/src/apps/ums.web-app/tests/profile-slice.spec.ts +++ b/src/apps/ums.web-app/tests/profile-slice.spec.ts @@ -1,3 +1,4 @@ +// @story FS-05 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/profile-state.spec.ts b/src/apps/ums.web-app/tests/profile-state.spec.ts index 91b63a11..0331ea2e 100644 --- a/src/apps/ums.web-app/tests/profile-state.spec.ts +++ b/src/apps/ums.web-app/tests/profile-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-05, FS-24, FS-40 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/role-state.spec.ts b/src/apps/ums.web-app/tests/role-state.spec.ts index 06c179ea..a63ec6f2 100644 --- a/src/apps/ums.web-app/tests/role-state.spec.ts +++ b/src/apps/ums.web-app/tests/role-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-17, FS-40 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/system-suite-actions.spec.ts b/src/apps/ums.web-app/tests/system-suite-actions.spec.ts index 4ff24891..b080b188 100644 --- a/src/apps/ums.web-app/tests/system-suite-actions.spec.ts +++ b/src/apps/ums.web-app/tests/system-suite-actions.spec.ts @@ -1,3 +1,4 @@ +// @story FS-04, FS-40 import { test, expect, type Page, type Locator } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/system-suite-modules.spec.ts b/src/apps/ums.web-app/tests/system-suite-modules.spec.ts index 34094f0a..e05f7e27 100644 --- a/src/apps/ums.web-app/tests/system-suite-modules.spec.ts +++ b/src/apps/ums.web-app/tests/system-suite-modules.spec.ts @@ -1,3 +1,4 @@ +// @story FS-04 import { test, expect, type Page, type Locator } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/system-suite-state.spec.ts b/src/apps/ums.web-app/tests/system-suite-state.spec.ts index 4aaa2473..2fdb90d6 100644 --- a/src/apps/ums.web-app/tests/system-suite-state.spec.ts +++ b/src/apps/ums.web-app/tests/system-suite-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-04 import { test, expect, type Page } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/tenant-state-toggle.spec.ts b/src/apps/ums.web-app/tests/tenant-state-toggle.spec.ts index e6a47591..df2e9387 100644 --- a/src/apps/ums.web-app/tests/tenant-state-toggle.spec.ts +++ b/src/apps/ums.web-app/tests/tenant-state-toggle.spec.ts @@ -1,3 +1,4 @@ +// @story FS-03, FS-40 import { test, expect } from '@playwright/test'; /** diff --git a/src/apps/ums.web-app/tests/user-account-state.spec.ts b/src/apps/ums.web-app/tests/user-account-state.spec.ts index fe488b01..a0c53d4f 100644 --- a/src/apps/ums.web-app/tests/user-account-state.spec.ts +++ b/src/apps/ums.web-app/tests/user-account-state.spec.ts @@ -1,3 +1,4 @@ +// @story FS-40 import { test, expect, type Page } from '@playwright/test'; import { borrarResiduo } from './helpers/limpieza';