test: add a real GPCM parameter-recovery test for fast-mlsirm - #452
test: add a real GPCM parameter-recovery test for fast-mlsirm#452seonghobae wants to merge 5 commits into
Conversation
Two TypeScript build errors on main (blocking every open PR's
"Frontend lint, test, build" check, including this repo's own review
bot's ability to approve them):
- App.tsx imported rememberOidcReturnUrl/returnUrlFromLocation from
oidcReturnUrl.ts but never called them -- the login button built its
own unsanitized returnUrl inline instead of using the safe helper
(oidcReturnUrl.ts's isSafeReturnUrl guard against an open-redirect-
shaped value) or persisting it as the sessionStorage/localStorage
fallback restoreOidcReturnUrl (already wired up on the callback side
in main.tsx) reads when the OIDC state round-trip drops it.
- The unauthenticated login screen unconditionally rendered
<AdminPanel accessToken={accessToken} /> when destination === "admin"
-- accessToken is string | undefined here (always undefined while
unauthenticated), a real type error, and the render was unreachable
through normal navigation (destination only changes via the
authenticated nav) -- dead code, removed.
uv run --frozen python -m pytest -q: 753 passed, 17 skipped.
pnpm run test: 140 passed. pnpm run lint / build: clean.
Simulates polytomous GPCM responses from known true item parameters and person thetas using Muraki (1993)'s additive category-intercept formula (fast_mlsirm ships no polytomous simulator, so it's implemented directly, matching PolytomousFit's own documented parameterization), fits them with the same fit_polytomous(..., model="gpcm") period_report.py's production code can select, and asserts recovered EAP thetas match true thetas within a literature-typical bound. Also notes cat_simulate_polytomous (a real CAT simulator over a fitted GRM/GPCM bank) exists as a concrete next step for the CAT part of the same gap entry.
|
Warning Review limit reached
Next review available in: 16 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (4)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
…k' into repair/pr452-frontend-root
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
The GRM recovery test and its measured RMSE/correlation numbers live in a separate, still-open PR and are not part of this branch. Only claim what this PR itself adds (test_fast_mlsirm_gpcm_recovery.py) and note the GRM test as still in progress elsewhere, so the doc can't be merged into a state where it cites a test file that does not exist in the repo. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5M79L945DMyMs3sg5yJ14
PR454's tree only adds tests/test_fast_mlsirm_fipc_recovery.py; the GRM/GPCM/CAT recovery tests live on separate open PRs (#451/#452/#453). Claiming "(Resolved)" with all four files here is false if this PR merges independently, so scope the claim to FIPC and note the other three are pending on their own PRs.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
docs/product-technical-gap-baseline.md claimed all three fast-mlsirm recovery tests (GRM, GPCM, CAT) exist and were "Mostly resolved," but this PR only adds the CAT test -- GRM (#451) and GPCM (#452) are still open, unmerged PRs. Scope the entry to what this PR actually ships and note the other two as in flight. test_fast_mlsirm_cat_recovery.py's MAX_MEAN_ITEMS_USED=20 bound was loose enough that a non-adaptive (random item order) run of the same fixture/seed also passes (measured mean_items_used ~14.97), so the test would not catch a silent regression that dropped the `adaptive` flag. Tighten the bound to 12, close to the real adaptive-run measurement (~8.7) and verified to fail the non-adaptive fallback case.
|
Reviewed the remaining informational thread. The login/AdminPanel changes are the shared OIDC root fix and are covered by their dedicated tests. @opencode-agent please independently review exact head |
|
Reviewed the remaining informational thread. The login/AdminPanel changes are the shared OIDC root fix and are covered by their dedicated tests. @opencode-agent please independently review exact head |
|
Closing as out of lane for LineageWeave. IRT/linking recovery tests belong in fast-mlsirm, not this repo. Unique product slice remains #258. Do not restack onto the GNB-four head. |
Pull request was closed
1 participant
Summary
period_report.py's production code fits GRM and GPCM and picks between them.fast_mlsirm's installed source directly (polytomous.py) rather than guessing at the GPCM formula:PolytomousFit's own docstring documents GPCM as "additive category intercepts" with a "Muraki step reparametrization" — the standard Muraki (1993) generalized partial credit model, distinct from GRM's cumulative-logistic thresholds.tests/test_fast_mlsirm_gpcm_recovery.pyimplements that exact formula (cumulative step logits, softmax over categories) to generate synthetic responses from known true item discriminations/steps and person thetas, fits via the realfit_polytomous(..., model="gpcm"), scores viascore_polytomous.RMSE < 0.55,correlation > 0.8).fast_mlsirm.cat_simulate_polytomousis a real computerized-adaptive-test simulator over a fitted GRM/GPCM item bank (Dodd, De Ayala & Koch, 1995) — directly relevant to the "CAT" part of the same gap-baseline entry. Noted as a concrete next step, not attempted this cycle (staying to one well-scoped addition per cycle).cat_simulate_polytomousexists and is ready to exercise).Test plan
uv run --frozen python -m pytest -q tests/test_fast_mlsirm_gpcm_recovery.py— 1 passeduv run --frozen python -m pytest -q(full backend suite) — 754 passed/17 skipped, no regressions (this branch doesn't yet include test: add a real GRM parameter-recovery test for fast-mlsirm #451's GRM test since that's unmerged, so the count reflects origin/main + this one new test)🤖 Generated with Claude Code