Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ jobs:
- name: Typecheck (tsc --noEmit)
run: npx nx run-many --target=typecheck --projects=app-web

# Informe, no puerta: hoy 14 de las 25 historias «usable» no tienen prueba que las nombre,
# así que bloquear haría nacer el gate en rojo — y un gate que nace en rojo se ignora. Sí
# bloquea si una spec se etiqueta con una historia que el tracker no declara usable (TD-012).
- name: Story coverage report (e2e ↔ functional stories)
run: npx nx run-many --target=cobertura-historias --projects=app-web

- name: Run Unit Tests
# `--code-coverage` no es un flag válido de vitest/dotnet (vitest usa `--coverage`).
# Se corre el target `test` (vitest run); la cobertura se gestiona con test:coverage.
Expand Down
37 changes: 29 additions & 8 deletions docs/architecture/technical-debt.md
Original file line number Diff line number Diff line change
Expand Up @@ -316,15 +316,36 @@ The write test that caused the 133 is **kept, deliberately, as a sentinel**: it

---

---

## [TD-012] Nothing Connects a Functional Story to the Test That Backs It

- **Status**: Confirmed
- **Status**: **Resolved — measurement in place** (2026-08-10). The gap it measures is real and open.
- **Severity**: Medium
- **Component**: `docs/governance/project*/functional-story-gap-tracker.md`, `src/apps/ums.web-app/tests/*.spec.ts`, `docs/architecture/traceability-matrix.md`
- **Description**: The gap tracker declares **25 stories "Implemented / usable"**. Nothing verifies that claim. There are 23 e2e spec files and **84 cases**, and exactly **one** names a story — `FS-26`, which is not among the 25. The traceability matrix maps FS → ADR → Enabler, never FS → test.
- **How it surfaced (2026-08-10)**: a verification pass run on request. All 25 stories' REST surfaces answer against a live API, so the backend is not the issue. The issue is that the three defects fixed that same day were **front-end** defects, invisible from the API: `FS-20`/`FS-13` saved `false` for every boolean parameter regardless of the switch, and `FS-05` submitted `undefined` as the user id. Three stories sat green in the tracker while broken.
- **Measured coverage**: by keyword, `FS-16`, `FS-38` and `FS-39` have **no e2e trace at all**; `FS-09` and `FS-15` have one file each. The rest have de-facto coverage under feature-named specs (`profile-panel`, `role-state`, `parameter-catalog-state`…), but it is inferred from filenames, not declared.
- **Impact**: the column reads as verification and is intent. Anything hanging off it — a release scorecard, a date commitment, a conversation with product — inherits that gap. And a real regression can sit in a green row indefinitely, which is exactly what happened.
- **Suggested resolution**: name the story in each e2e case — `test('FS-05 · assign template to profile', …)` — and have CI assert that every story marked usable has at least one passing case. Same shape as the fix in [TD-004](#td-004): it adds no work, and it makes the claim impossible to hold while false.
- **Related**: the same disease as TD-002, TD-003, TD-010 and G-066 — a document asserting a state nobody checks against the code. This is its last known instance in this repository.
- **Component**: `docs/governance/project*/functional-story-gap-tracker.md`, `src/apps/ums.web-app/tests/*.spec.ts`
- **Description**: The tracker declared **25 stories "Implemented / usable"** and nothing verified it. 23 e2e spec files, 84 cases, and exactly one named a story — `FS-26`, not among the 25. The traceability matrix maps FS → ADR → Enabler, never FS → test.
- **How it surfaced**: three of those 25 were genuinely broken on 2026-08-10 — `FS-20`/`FS-13` saved `false` for every boolean parameter, `FS-05` submitted `undefined` as the user id — and all three sat green here. They were front-end defects, invisible from the API, so a REST sweep would never have found them.

### Resolution

Each spec declares the stories it answers to in its first line (`// @story FS-05, FS-24`), and `scripts/verificar-cobertura-historias.mjs` derives coverage **from the code**, comparing it against the tracker's own table. A separate mapping file was rejected: it would have drifted exactly like everything else this register records.

`npm run cobertura:historias`, wired into CI ahead of the unit tests.

**First measurement — 11 of 25 covered, 14 not:**

| | Stories |
| :-- | :-- |
| **Covered** (11) | FS-01, FS-02, FS-03, FS-04, FS-05, FS-13, FS-14, FS-17, FS-20, FS-24, FS-40 |
| **No e2e naming them** (14) | FS-06, FS-07, FS-08, FS-09, FS-10, FS-11, FS-15, FS-16, FS-18, FS-19, FS-21, FS-22, FS-38, FS-39 |

Three specs answer to no story — `authorization-ui`, `navigation`, `profile-panel` — legitimately: they test UI mechanics, not business outcomes. `profile-panel` is worth naming: the filename sounds like FS-05 and every case is avatar and drawer behaviour. Mapping by filename would have credited FS-05 with coverage it does not have, which is why this was done case by case.

### Why it reports instead of blocking

With 14 of 25 uncovered, a hard gate would be red from birth, and a gate that is red from birth gets ignored. It **does** fail on one condition: a spec tagged with a story the tracker does not declare usable — a labelling error, and a test claiming to cover something unrecognised is worse than no label.

Promoting it to a gate (`--estricto`) is a decision for when the 14 are closed, not before.

- **The underlying gap stays open**: 14 stories are declared usable with nothing exercising them end to end. What changed is that the number is measured on every CI run instead of assumed.

1 change: 1 addition & 0 deletions src/apps/ums.web-app/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
"lint": "eslint .",
"lint:fix": "eslint . --fix",
"typecheck": "tsc --noEmit -p tsconfig.app.json && tsc --noEmit -p tsconfig.node.json",
"cobertura:historias": "node scripts/verificar-cobertura-historias.mjs",
"format": "prettier --write .",
"format:check": "prettier --check .",
"test": "vitest run",
Expand Down
6 changes: 6 additions & 0 deletions src/apps/ums.web-app/project.json
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@
"options": {
"cwd": "apps/ums.web-app"
}
},
"cobertura-historias": {
"command": "npm run cobertura:historias",
"options": {
"cwd": "apps/ums.web-app"
}
}
}
}
78 changes: 78 additions & 0 deletions src/apps/ums.web-app/scripts/verificar-cobertura-historias.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
#!/usr/bin/env node
/**
* verificar-cobertura-historias.mjs — ¿qué historia respalda cada prueba e2e? (TD-012)
*
* El tracker declara historias «Implemented / usable» y hasta ahora nada lo comprobaba: 84 casos
* e2e y uno solo nombraba una historia. Tres historias podían estar rotas —y lo estuvieron el
* 2026-08-10— sin que la fila dejara de estar en verde.
*
* La cobertura se deriva del CÓDIGO, no de un documento aparte: cada spec declara a qué historias
* responde con un `// @story FS-NN` en su primera línea. Un fichero de mapeo se habría
* desincronizado igual que se desincronizó todo lo demás.
*
* Salida: informe, no puerta. Hoy 14 de las 25 no tienen cobertura, así que bloquear sería nacer
* en rojo — y una puerta que nace en rojo se aprende a ignorar. Se bloquea cuando `--estricto`.
*/
import { readFileSync, readdirSync } from 'node:fs';
import { join, dirname } from 'node:path';
import { fileURLToPath } from 'node:url';

const AQUI = dirname(fileURLToPath(import.meta.url));
const DIR_PRUEBAS = join(AQUI, '..', 'tests');
const TRACKER = join(AQUI, '..', '..', '..', '..', 'docs', 'governance', 'project', 'functional-story-gap-tracker.md');

/** Historias que el tracker declara implementadas y usables. */
function historiasDeclaradasUsables() {
const md = readFileSync(TRACKER, 'utf8');
// La fila de la TABLA, no la prosa: el propio tracker lleva una nota que cita esa etiqueta
// entre comillas, y buscarla por texto suelto engancha el párrafo en vez de los datos.
const fila = md.split('\n').find(l => l.startsWith('|') && l.includes('Implemented / usable'));
if (!fila) throw new Error('No se encontró la fila «Implemented / usable» en la tabla del tracker.');
return [...new Set(fila.match(/FS-\d+/g) ?? [])];
}

/** Historias que cada spec declara cubrir, leídas del propio fichero. */
function coberturaDeclaradaEnPruebas() {
const porHistoria = new Map();
const sinHistoria = [];
for (const f of readdirSync(DIR_PRUEBAS).filter(n => n.endsWith('.spec.ts'))) {
const primera = readFileSync(join(DIR_PRUEBAS, f), 'utf8').split('\n', 1)[0];
const historias = primera.startsWith('// @story') ? (primera.match(/FS-\d+/g) ?? []) : [];
if (historias.length === 0) sinHistoria.push(f);
for (const h of historias) porHistoria.set(h, [...(porHistoria.get(h) ?? []), f]);
}
return { porHistoria, sinHistoria };
}

const usables = historiasDeclaradasUsables();
const { porHistoria, sinHistoria } = coberturaDeclaradaEnPruebas();

const conCobertura = usables.filter(h => porHistoria.has(h));
const sinCobertura = usables.filter(h => !porHistoria.has(h));

console.log(`\nHistorias declaradas usables: ${usables.length}`);
console.log(` con prueba e2e que las nombra: ${conCobertura.length}`);
console.log(` SIN ninguna: ${sinCobertura.length}\n`);

for (const h of conCobertura) console.log(` ✓ ${h} ${porHistoria.get(h).join(', ')}`);
if (sinCobertura.length) {
console.log('\n Sin cobertura declarada:');
for (const h of sinCobertura) console.log(` ✗ ${h}`);
}
if (sinHistoria.length) {
console.log(`\n Specs sin @story (no responden a ninguna historia): ${sinHistoria.join(', ')}`);
}

// Una historia etiquetada que el tracker NO declara usable es un error de etiqueta, y ese sí
// bloquea siempre: significa que una prueba afirma cubrir algo que el tracker no reconoce.
const desconocidas = [...porHistoria.keys()].filter(h => !usables.includes(h));
if (desconocidas.length) {
console.error(`\nERROR: specs etiquetadas con historias que el tracker no declara usables: ${desconocidas.join(', ')}`);
process.exit(1);
}

if (process.argv.includes('--estricto') && sinCobertura.length) {
console.error(`\nERROR (--estricto): ${sinCobertura.length} historias usables sin prueba.`);
process.exit(1);
}
console.log('');
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/app-configuration-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-13
import { test, expect, type Page, type APIResponse } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/auth.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-01
import { test, expect } from '@playwright/test';

test.describe('Authentication Flow', () => {
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/business-flows.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-02, FS-14
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-14
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/feature-flag-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-40
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/identity-provider.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-03
import { test, expect, type Page, type APIResponse } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/login.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-01
import { test, expect } from '@playwright/test';

test.describe('Login Flow', () => {
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/mvp-creation-chain.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-02, FS-04
import { test, expect, Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/mvp-flows.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-03, FS-04
import { test, expect } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/parameter-catalog-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-20
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/parameter-values.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-13
import { test, expect, type Page, type APIResponse } from '@playwright/test';

/**
Expand Down
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-02
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/profile-slice.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-05
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/profile-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-05, FS-24, FS-40
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/role-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-17, FS-40
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/system-suite-actions.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-04, FS-40
import { test, expect, type Page, type Locator } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/system-suite-modules.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-04
import { test, expect, type Page, type Locator } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/system-suite-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-04
import { test, expect, type Page } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/tenant-state-toggle.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-03, FS-40
import { test, expect } from '@playwright/test';

/**
Expand Down
1 change: 1 addition & 0 deletions src/apps/ums.web-app/tests/user-account-state.spec.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
// @story FS-40
import { test, expect, type Page } from '@playwright/test';
import { borrarResiduo } from './helpers/limpieza';

Expand Down
Loading