docs: mark the gap-baseline omni-modal image entry resolved - #441
docs: mark the gap-baseline omni-modal image entry resolved#441seonghobae wants to merge 2 commits into
Conversation
lineageweave/image_content.py's OpenAiCompatibleVisionClient already runs every embedded image through a vision-capable contextual- orchestrator model for OCR, layout/object/relationship captioning, and tagging, and post_content_normalization.py inlines that output into the post body before chunking so it reaches the normal embedding pipeline. The doc's "text-only embedding fails on images" claim predates this and was misleading future work toward rebuilding an already-wired capability. Left a narrower, accurate follow-up note about font-size/color not being explicit structured fields.
|
Warning Review limit reached
Next review available in: 58 minutes Limit details: You’ve used the included review currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Closing after a hunk-level audit against the integration train (feat/board-source-detail-state-filter, PR #490): the login return-URL fix this PR carries is in the train verbatim at all three signinRedirect sites (App.tsx:6354-6412), the stray unauthenticated AdminPanel render is already removed there, and the train's App.test.tsx asserts the sessionStorage+localStorage round-trip exactly (lines 64-67, stronger than this PR). The remaining hunk edits docs/product-technical-gap-baseline.md's OLD bullet structure, which the train fully restructured — this diff can no longer merge — and its resolution claim is contradicted by the train doc's newer, quantified audit of the same topic, which still tracks it as open/partial with evidence. The claim text remains in this PR's history; if any part of it is verifiably true today, the right fix is a fresh edit against the train's live doc, not this stale diff. |
Pull request was closed
Summary
docs/product-technical-gap-baseline.mdclaimed "Current text-only embedding fails on images. Omni-modal LLM processing is required for images to capture layout, font size, colors, and spatial meaning."lineageweave/image_content.py'sOpenAiCompatibleVisionClientalready sends every embedded image to a vision-capable model (via the samecontextual-orchestratorgateway as every other LLM channel —_vision_client()/orchestrator_vision_client()inbackend/app/main.py), asking for OCR text plus a caption explicitly covering "visible layout, objects, relationships, directions, measurements, and labels", plus tags.post_content_normalization.pyinlines that caption+OCR at the image's original position in the post body before chunking, so it reaches the normal embedding pipeline rather than being silently dropped as text-only.(Resolved), matching the doc's existing convention, with an honest narrower follow-up note: the caption prompt doesn't explicitly request font size/color as separate structured fields (only incidentally, when visually salient) — a real but much smaller gap than the entry originally described.Test plan
🤖 Generated with Claude Code