Skip to content

feat(job-analysis): add governed qualification-rule review - #104

Open
seonghobae wants to merge 13 commits into
developfrom
feat/job-qualification-rule-review
Open

feat(job-analysis): add governed qualification-rule review#104
seonghobae wants to merge 13 commits into
developfrom
feat/job-qualification-rule-review

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 23, 2026

Copy link
Copy Markdown
Contributor

Buyer-visible gap

Protected develop@9e3e4847510e1e612b48474ba42b177b8ed824df models Job Analysis Task/FJA/KSAO evidence and the PRD requires qualification rules, but it has no bounded governance evidence proving which Job Analysis evidence supports a proposed qualification rule before that rule can influence recruiting or selection.

This Orgmetra-only slice adds a human-reviewed, PII-minimized qualification-rule review boundary. It does not evaluate candidates, reject applicants, mutate Job/Job Analysis, or write any foreign CWL repository.

Test-first repair sequence

  • RED contract f06498079027b0f381beaf500ba7487ddcd6e555 introduced the package contract, adversarial regressions, exact-head workflow, ADR, APA-7 doctoring, traceability, README and CHANGELOG while production review.py was absent. No predecessor/superseded run is promoted as current evidence.
  • The current branch implements JobQualificationRuleReviewPacket with exact runtime primitives, authoritative non-sentinel Job/Job Analysis references, packet-owned UUIDv4 artifact/actor correlations, lower-case SHA-256 Job Analysis/rule/Task/KSAO/source provenance, controlled non-sensitive rule categories and reasons, business-effective date, explicit evidence version, distinct requester/reviewer, human review time and owner-generated system-recorded UTC issuance time.
  • The current exact head d92ac4cb798b3bd32b632c0ab677c03f944070e4 additionally fixes the malformed-but-correctly-namespaced Job reference boundary: job_record:not-a-uuid fails closed rather than escaping the governed validation path.

Current contract

Canonical evidence is fixed to job_qualification_rule_review, mandatory human review, reviewed_for_authoritative_resolution, and not_authorized_for_candidate_or_employment_decision. It excludes candidate/person PII, candidate qualification outcomes, assessment/cut scores, compensation, raw qualification-rule text, prompts and model output.

Before an authoritative rule is persisted or used in recruiting/selection, the host must re-resolve the exact tenant/Job/Job Analysis snapshot, qualification-rule artifact, Task/KSAO/source provenance and accountable reviewer authority at the relevant business-effective coordinate, then preserve immutable audit/outbox evidence. The packet itself grants no candidate or employment-decision authority.

Trust-bearing text, dates, timestamps and evidence versions require exact runtime primitives; Nil/Max UUID sentinels, malformed/noncanonical references, malformed SHA-256, actor overlap, unreviewed categories/reasons, future human review relative to issuance, mutated governance constants and post-issuance evidence mutation fail closed. Canonical exports are generated from a verified evidence snapshot against the creation-time issuance digest.

Research boundary

OPM job-analysis and qualification-policy guidance plus the Uniform Guidelines on Employee Selection Procedures are recorded as design evidence only. This package makes no legal-compliance, validity, qualification, or certification determination.

Exact-current-head evidence

Base: develop@9e3e4847510e1e612b48474ba42b177b8ed824df
Head: d92ac4cb798b3bd32b632c0ab677c03f944070e4

Every applicable exact-current-head workflow is terminal GREEN:

  • Job Qualification Rule Review Quality 32671485522 — success
  • Foundation CI 32671485458 — success
  • Recovery Rehearsal Quality 32671485471 — success
  • SAST Semgrep 32671485492 — success
  • Security Scan 32671485484 — success

Fresh review state has one COMMENTED Devin review and two unresolved informational threads: identity-based hashing is deliberately required by the process-local weak issuance registry so field mutation cannot change lookup identity, and the review/issuance chronology guard intentionally fails closed when reviewer time is ahead of the issuing host clock. Neither thread identifies a current authorization, privacy, data-integrity, or canonical-evidence bypass, so they remain unresolved rather than being falsely marked addressed. There is no qualifying independent non-author APPROVE and no CHANGES_REQUESTED review. GitHub reports the PR open, ready-for-review and mergeable.

Merge governance

Ready-for-review is not merge authorization. Keep unmerged until a qualifying independent non-author approval exists on this unchanged head and issue #89's enforceable develop protection defect is actually repaired. Immediately before any future merge, refetch exact head, live base, reviews, unresolved threads, effective rules/protection and exact-current-head checks and use expected-head protection only if every live gate is satisfied. Do not self-approve, weaken or simulate protection, manufacture passing evidence, or transfer predecessor checks/reviews.

@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 46 minutes.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: fb101d6b-3fe5-4b93-a308-deaa2e2ba964

📥 Commits

Reviewing files that changed from the base of the PR and between 9e3e484 and 79adb79.

📒 Files selected for processing (10)
  • .github/workflows/job-qualification-rule-review-quality.yml
  • docs/adr/0104-governed-job-qualification-rule-review.md
  • docs/doctoring/job-qualification-rule-review-references.md
  • docs/traceability/job-qualification-rule-review.md
  • packages/job-qualification-rule-review/CHANGELOG.md
  • packages/job-qualification-rule-review/README.md
  • packages/job-qualification-rule-review/pyproject.toml
  • packages/job-qualification-rule-review/src/orgmetra_job_qualification_rule_review/__init__.py
  • packages/job-qualification-rule-review/src/orgmetra_job_qualification_rule_review/review.py
  • packages/job-qualification-rule-review/tests/test_review.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

Current-head correction: production implementation is now present at 4c42b422fbca9d4bc79c04b67f778632d0be2aca. The earlier body section describing f0649807… as the current RED head is stale; that RED contract was superseded before terminal execution and is not passing evidence. Exact-current-head Job Qualification Rule Review, Foundation, Recovery, SAST, and Security workflows are materialized but queued, so this PR remains Draft and no predecessor result is promoted. The current source binds reviewed Job/Job-Analysis/rule-artifact Task/KSAO/source provenance, generates system-recorded UTC time inside the owner boundary, keeps canonical evidence value-minimized, and remains non-authoritative for downstream personnel actions.

Copy link
Copy Markdown
Contributor Author

Current-head evidence update: d92ac4cb798b3bd32b632c0ab677c03f944070e4 adds a realistic malformed-but-correctly-namespaced UUID regression (job_record:not-a-uuid) so the shared reference parser's fail-closed exception path is exercised rather than relying on only wrong-namespace/sentinel/version cases. Production behavior is unchanged. The prior head's queued results do not transfer, and no hosted run has materialized yet for d92ac4c…; under the repository evidence policy this remains non-passing and #104 remains Draft. Protected base remains develop@9e3e4847510e1e612b48474ba42b177b8ed824df.

@seonghobae
seonghobae marked this pull request as ready for review August 23, 2026 23:09

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Open in Devin Review

return json.dumps(payload, sort_keys=True, separators=(",", ":"), ensure_ascii=True)


@dataclass(frozen=True, slots=True, weakref_slot=True, eq=False, repr=False)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Tamper detection relies on identity-based hash

The packet uses eq=False, so hashing and equality fall back to object identity. This is what lets the weak-key issuance store still find the original digest after a field is mutated, making the post-issuance tamper check work (review.py:298-306). Switching to eq=True or a field-based hash would break that guarantee.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment on lines +255 to +256
if recorded_at < reviewed_at:
raise ValueError("recorded_at cannot precede reviewed_at")

@devin-ai-integration devin-ai-integration Bot Aug 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Future-review guard tied to construction wall clock

The only guard against a future review time is recorded_at < reviewed_at (review.py:255), where recorded_at is now() at construction. Clock skew that makes a reviewer timestamp slightly ahead of the issuing host will reject an otherwise valid review.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

@seonghobae
seonghobae marked this pull request as draft August 24, 2026 15:41
@seonghobae
seonghobae marked this pull request as ready for review August 24, 2026 15:41
@seonghobae seonghobae closed this Aug 24, 2026
@seonghobae seonghobae reopened this Aug 24, 2026
@seonghobae

Copy link
Copy Markdown
Contributor Author

@opencode-agent Please review the current unchanged head against protected develop. Local exact-head verification: all owned package suites pass at 100% statement/branch coverage.

…n-subclassable

Strix OpenAI-direct scan of head d92ac4c surfaced one MEDIUM finding:
a hostile in-process subclass could override _validated_payload to
bypass all trust-bearing field validation.

Repair: __init_subclass__ now raises TypeError for every subclass, so
the trust boundary is non-overridable by construction. Regression
coverage asserts subclass definition fails closed at class-definition
time and base-class canonical evidence stays stable per issued instance.
55 tests, 100% owned statement/branch coverage.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

Open in Devin Review


assert first.canonical_json() == first.canonical_json()
assert first.sha256_digest() == first.sha256_digest()
assert second.sha256_digest() != first.sha256_digest()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Digest inequality test relies on clock granularity

test_valid_packets_still_canonicalize_and_hash_deterministically asserts two packets built from identical inputs hash differently. The only differing field is recorded_at from datetime.now(timezone.utc) (review.py). If two constructions share a clock tick the digests match and the test fails. Microsecond resolution makes this practically unreachable on Linux, but the assertion depends on timer granularity, not a deterministic difference.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


with pytest.raises(TypeError, match="must not be subclassed"):

class ForgedPacket(JobQualificationRuleReviewPacket):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant