feat(ask): cite persisted image evidence for cited posts - #419
Merged
Conversation
An Ask answer citing a post whose evidence actually came from an
embedded picture (a screenshot, a diagram) read as an unmarked text
claim -- no way to tell the citation was image-sourced. Raw image bytes
are never sent to the client anywhere in this codebase (only persisted
OCR/caption/tags -- see lineageweave/image_content.py); this reuses that
same never-raw-bytes description GET /api/posts/{id}/content already
renders, scoped to already-cited posts.
Backend: cited_post_images (backend/app/post_chat_ingestion.py) reads
post_content_image/post_content_image_tag for the cited post ids, no
extra ABAC check needed (cited_post_ids only ever come from
gather_global_chat_sources's already-authorized source set, same trust
boundary cited_post_evidence/cited_post_summaries rely on). Wired into
POST /api/ask as a new cited_post_images response field.
Frontend: AskAgentPanel renders an "Image evidence" line under a cited
post when present, with the persisted caption and OCR text. Adds the ko
/ zh / ja / vi translations for the two new strings.
Part of the Ask Agent temporal/lineage/evidence goal (checkpoint 3 of 4).
|
Warning Review limit reached
Next review available in: 46 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
📝 WalkthroughWalkthroughAsk Agent 응답에 인용 게시물의 이미지 근거를 추가했습니다. 프런트엔드는 이미지와 텍스트 근거를 접근 가능한 팝업에서 표시합니다. 백엔드 조회, API 타입, 통합 테스트, Storybook 상태 및 관련 번역을 추가했습니다. ChangesAsk Agent 근거 표시
Estimated code review effort: 4 (Complex) | ~45 minutes Merge Risk: 🔵 Low · up to Cited image evidence can display a blank caption when the persisted caption is empty or whitespace instead of showing “Untitled image.” This is a bounded UI correctness issue; the change is otherwise mergeable with explicit owner follow-up. Sequence Diagram(s)sequenceDiagram
participant AskAgent as Ask Agent
participant AskAPI as /api/ask
participant ImageQuery as cited_post_images
participant App as App.tsx
participant Popup as AskEvidenceLayerPopup
AskAgent->>AskAPI: 질문 전송
AskAPI->>ImageQuery: 인용 게시물 ID 전달
ImageQuery-->>AskAPI: 이미지 근거 반환
AskAPI-->>App: cited_posts 및 cited_post_images 반환
App->>Popup: 선택한 게시물의 사실 및 이미지 전달
Popup-->>App: 닫기 또는 원문 게시물 열기
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This was referenced Aug 22, 2026
* feat(ask): show cited-post evidence in a Layer Popup Reading an Ask answer's evidence today means either scanning the inline fact list or leaving the answer entirely to open the full post popup. Add AskEvidenceLayerPopup (frontend/src/components/), a focused modal layer -- opened via a new "View evidence" button per citation -- showing that post's text evidence facts (checkpoint 3's cited_post_evidence) and image evidence (checkpoint 3's cited_post_images) without navigating away from the answer. Proper dialog semantics: role="dialog", aria-modal, Escape-to-close, backdrop-click-to-close, initial focus on the panel -- stricter accessibility than the existing PostDetailPopup, which has none of these. Extracted chatEvidenceKindLabel into evidenceKindLabels.ts so both App.tsx and the new component share one label map instead of drifting. Stacked on #419 (checkpoint 3): the popup's image-evidence section needs that PR's cited_post_images field to be meaningful. Part of the Ask Agent temporal/lineage/evidence goal (checkpoint 4 of 4). * test(ask): cover evidence dialog edge cases * fix(ask): contain evidence dialog focus * docs(storybook): cover blank evidence caption * docs(ask): trace modal accessibility standard * docs(storybook): inventory Ask evidence layer * docs(changelog): record Ask evidence layer * fix(changelog): restore historical entries * test(ask): cover modal exit focus and source transition * fix(ask): restore focus when evidence modal exits * fix(frontend): drop stray pre-login AdminPanel, restore return-URL persistence The pre-login screen rendered AdminPanel with a possibly-undefined accessToken (a TS6192/TS2322 build break) and never used the persisted return-URL helpers on the login redirect. Same fix as LineageWeave#456 on main, applied here since this stack predates that fix.
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
seonghobae
enabled auto-merge (squash)
August 23, 2026 06:41
image.caption ?? t("Untitled image") only substitutes for null/undefined,
so a persisted empty-string caption rendered a blank label ("Image
evidence: — ..."). Match the blank-aware fallback already used in
AskEvidenceLayerPopup.tsx (image.caption?.trim() ? image.caption : ...)
so the same evidence renders consistently in both surfaces.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B5M79L945DMyMs3sg5yJ14
seonghobae
enabled auto-merge (squash)
August 23, 2026 07:50
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
Contributor
Author
|
Fixed empty image citations at exact head |
seonghobae
enabled auto-merge (squash)
August 23, 2026 08:18
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
seonghobae
enabled auto-merge
August 24, 2026 01:25
seonghobae
added a commit
that referenced
this pull request
Aug 24, 2026
) * test(e2e): add a Playwright harness for the Ask Agent capabilities No Playwright config existed despite the package already being a devDependency. Add playwright.config.ts (points at the running docker-compose stack, not a Playwright-managed dev server -- the app needs Postgres/Keycloak/Valkey/orchestrator alongside it), a real Keycloak-login helper (drives the actual OIDC redirect form with the synthetic demo.analyst credentials, not a token injected into storage), a validated smoke spec, and ask-agent.spec.ts covering all four Ask Agent capabilities (relative-time retrieval #415, multi-lineage graphs #418, image citation #419, Layer Popup #420). The login flow is verified passing against a live stack right now. ask-agent.spec.ts needs #415/#418/#419/#420 merged and the images rebuilt from main before it can pass -- verified during development that the currently-running ad-hoc stack is built from an unrelated, more advanced branch (its own conversation-history UI), not main, so it cannot validate this spec; that requires a proper CI/deployment rebuild, out of this checkpoint's scope. Part of the Ask Agent temporal/lineage/evidence goal (checkpoint 5 of 6 -- e2e harness). * fix(frontend): repair the inherited login/admin-panel build break Two TypeScript build errors on main (blocking every open PR's "Frontend lint, test, build" check, including this repo's own review bot's ability to approve them): - App.tsx imported rememberOidcReturnUrl/returnUrlFromLocation from oidcReturnUrl.ts but never called them -- the login button built its own unsanitized returnUrl inline instead of using the safe helper (oidcReturnUrl.ts's isSafeReturnUrl guard against an open-redirect- shaped value) or persisting it as the sessionStorage/localStorage fallback restoreOidcReturnUrl (already wired up on the callback side in main.tsx) reads when the OIDC state round-trip drops it. - The unauthenticated login screen unconditionally rendered <AdminPanel accessToken={accessToken} /> when destination === "admin" -- accessToken is string | undefined here (always undefined while unauthenticated), a real type error, and the render was unreachable through normal navigation (destination only changes via the authenticated nav) -- dead code, removed. uv run --frozen python -m pytest -q: 753 passed, 17 skipped. pnpm run test: 140 passed. pnpm run lint / build: clean. * test(frontend): keep Playwright specs out of Vitest * fix(e2e): stop Ask Agent lineage/image tests silently skipping their assertions The git-branch-lineage and image-citation Ask Agent tests wrapped their only real assertions in an `if (count > 0)` guard, so a run where the LLM's answer happened not to produce that citation shape would report PASS without exercising the behavior the test claims to cover. Assert the precondition itself so the test fails loudly instead. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B5M79L945DMyMs3sg5yJ14 * fix(e2e): stabilize Ask Agent locale and multi-match checks --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
seonghobae
added a commit
that referenced
this pull request
Aug 24, 2026
) * docs: record ADRs for the Ask Agent temporal/lineage/evidence goal Four new ADRs, one per checkpoint of the Ask Agent temporal/lineage/ evidence goal: - 0119: Korean relative-time expression resolution (#415) - 0120: multi-thread Event Lineage graphs in Ask answers (#418) - 0121: image citation without a new image-serving surface (#419) - 0122: the evidence Layer Popup (#420) Update CHANGELOG.md's Unreleased section and add an "Ask Agent Gaps" section to docs/product-technical-gap-baseline.md marking all four gaps (plus e2e coverage, #421) resolved, following that file's existing "(Resolved)" convention. Part of the Ask Agent temporal/lineage/evidence goal (checkpoint 6 of 6 -- documentation). * docs: renumber Ask Agent ADRs 0119-0122 to 0150-0153 Cross-session coordination surfaced a widespread ADR-numbering collision: at least ten numbers between 0119 and 0143 are independently claimed by concurrent unmerged branches across other sessions (0127/0128/0129/0131/0132 each claimed 2-4x, per `git log --all --diff-filter=A -- docs/adr`). This PR's own 0119-0122 was a three-way collision (also claimed by the TEPP topic-lineage PR and a quantity-superscript PR). Since this PR only holds 4 ADRs against another's 14 (0119-0132), renumbering here is the smaller diff. Moved clear of every number seen across all branches (highest observed: 0143), leaving buffer room. No content changes -- only the ADR number in each file's title, their mutual cross-references, and every CHANGELOG.md / gap-baseline.md citation of the old numbers. * docs: refresh product technical gap baseline * docs: record armed acceptance queue * fix(frontend): repair the inherited login/admin-panel build break Two TypeScript build errors on main (blocking every open PR's "Frontend lint, test, build" check, including this repo's own review bot's ability to approve them): - App.tsx imported rememberOidcReturnUrl/returnUrlFromLocation from oidcReturnUrl.ts but never called them -- the login button built its own unsanitized returnUrl inline instead of using the safe helper (oidcReturnUrl.ts's isSafeReturnUrl guard against an open-redirect- shaped value) or persisting it as the sessionStorage/localStorage fallback restoreOidcReturnUrl (already wired up on the callback side in main.tsx) reads when the OIDC state round-trip drops it. - The unauthenticated login screen unconditionally rendered <AdminPanel accessToken={accessToken} /> when destination === "admin" -- accessToken is string | undefined here (always undefined while unauthenticated), a real type error, and the render was unreachable through normal navigation (destination only changes via the authenticated nav) -- dead code, removed. uv run --frozen python -m pytest -q: 753 passed, 17 skipped. pnpm run test: 140 passed. pnpm run lint / build: clean. --------- Co-authored-by: seonghobae <seonghobae@users.noreply.github.com>
seonghobae
added a commit
that referenced
this pull request
Aug 24, 2026
…-source-detail-state-filter origin/main moved 58 commits ahead mid-session, including a properly reviewed and tested implementation of the same evidence-layer-popup feature this branch had manually rebuilt (#419). Re-merged to pick it up rather than ship a parallel, untested version. Resolved 7 conflicts: - backend/app/post_chat_ingestion.py: combined this branch's ABAC/candidate-id filtering with main's Korean relative-time date-range filter on the same source_post query; combined both sides' import additions (tepp_client + temporal_expressions). - backend/app/lineage_ingestion.py: combined this branch's isolation_reason (ADR 0143) with main's include_isolated parameter and new lineage_graphs_for_posts merge function -- both needed together, not alternatives. - backend/app/main.py: added the lineage_graph computation (from main's dropped PR #418) into this branch's persist_turn-based ask_agent flow, using response["cited_post_ids"] so it works for both the sources-empty and populated-sources paths. - frontend/src/App.tsx: kept this branch's working multi-turn (exchanges.map()) Ask Agent implementation wholesale over main's incompatible pre-refactor single-answer fragment; added the missing <LineageDag> rendering for response.lineage_graph per exchange, and kept the richer TenantConfig AdminPanel wiring already established. - tests/test_lineage_ingestion.py, tests/test_global_ask_sources.py, CHANGELOG.md: reconstructed interleaved additive conflicts as complete, non-overlapping test functions / changelog entries from both sides. Also fixed real bugs surfaced along the way: - cited_post_images() (backend/app/post_chat_ingestion.py) queried the pre-rename image.caption column; migration 0104 renamed it to image_caption. Fixed the SQL and row mapping, matching how /api/posts/{id}/content already does it. - Two tenant-settings tests asserted the old single-brandName response shape instead of the current {brandName, systemName, copyrightYear, copyrightHolder} contract. - A stale test fixture in tests/test_post_chat_ingestion.py still keyed its fake DB rows by the pre-rename "caption" column. - Fixed a duplicate ADR 0119 number (retire-buyer-terminology -> 0168; leftover-map-two-dimensional-distance already owned 0119). - A LineageDag.test.tsx role="group" vs a newly-merged App.test.tsx role="img" mismatch: kept role="group" (this branch's existing, better-tested LineageDag.tsx choice, verified by 4 passing assertions) and updated the one new test instead. Verified: tests/ (1037 passed, 11 skipped), backend/tests/ (156 passed, 5 skipped -- orchestrator-gated), frontend (469 passed), tsc -b clean. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NpWw9SnPBaemFZmW3fdTVM
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Buyer-visible gap
An Ask answer citing a post whose evidence actually came from an
embedded picture (a screenshot, a diagram) read as an unmarked text
claim -- the reader had no way to tell the citation was image-sourced
rather than drawn from the post's written body.
Change
Raw image bytes are never sent to the client anywhere in this codebase
-- only the persisted OCR text, caption, and tags
(
lineageweave/image_content.py). This reuses that same never-raw-bytesdescription
GET /api/posts/{id}/contentalready renders, scoped toposts the Ask answer already cited:
cited_post_images(backend/app/post_chat_ingestion.py) readspost_content_image/post_content_image_tagfor the cited post ids.No extra ABAC check needed:
cited_post_idsonly ever come fromgather_global_chat_sources's already-authorized source set -- thesame trust boundary
cited_post_evidence/cited_post_summariesalready rely on.
POST /api/askas a newcited_post_imagesresponsefield.
AskAgentPanelrenders an "Image evidence" line under a cited postwhen present, with the persisted caption and OCR text.
UI strings ("Image evidence", "Untitled image").
Part of a larger goal: Ask Agent should also understand Korean
relative-time expressions (#415), render cited lineage as git-branch-style
graphs (#418), and show evidence in a Layer Popup -- separate
checkpoints/PRs, independently based off
main.Verification
uv run --frozen python -m pytest -q-- 755 passed, 17 skipped(pre-existing, live-Keycloak/orchestrator-only integration tests).
pnpm run test(frontend) -- 142 passed, including 2 new cases: acited image renders its caption/OCR text, and no image line appears
when the answer cites no image.
pnpm run lint/tsc -b-- no new warnings/errors; the twopre-existing
App.tsxerrors are the unrelatedmainbreak alreadytracked against feat: show persisted image-region locations (v2.12.8) #405.
No real records, identifiers, or provider credentials are included.
Summary by CodeRabbit
새 기능
개선 사항
테스트