Skip to content

Support legacy Office images and optional formula extraction - #1280

Merged
Paul Lizer (paullizer) merged 1 commit into
Developmentfrom
paullizer-content-understanding-extraction
Aug 18, 2026
Merged

Support legacy Office images and optional formula extraction#1280
Paul Lizer (paullizer) merged 1 commit into
Developmentfrom
paullizer-content-understanding-extraction

Conversation

@paullizer

Copy link
Copy Markdown
Collaborator

Refs #1277

Two gaps surfaced while answering questions about equation and legacy-format coverage. Both are verified fixes, not speculative.

1. Images in legacy .doc and .ppt are now analyzed

Embedded image analysis covered only DOCX and PPTX. Legacy Office files are OLE compound documents rather than zip packages, so there is no word/media part to enumerate — the extractor found nothing and the files were excluded outright.

Their pictures and embedded equation previews are still stored as intact metafile blobs, so they are now carved out by signature using the length recorded in the metafile's own header, then rasterized and analyzed like any other embedded image.

Validation is deliberately strict — record type, signature position, and a declared length that fits inside the remaining bytes — so a coincidental byte sequence is not mistaken for an image. There is a test that plants a decoy " EMF" sequence and asserts it is rejected while a real metafile in the same buffer is still carved byte-exactly.

Verified against a synthetic legacy container built from two of the real diagrams plus a duplicate:

diagnostics: {'candidates': 3, 'analyzed': 2, 'skipped': 1, 'skipped_reasons': {'duplicate_image': 1}}
  embedded_1.emf 989x831  27 labels
  embedded_2.emf 1285x831 38 labels

Duplicate collapsing and the per-document cap still apply, and a file that is neither zip nor OLE yields nothing rather than raising.

2. Optional mathematical formula extraction

Equations were never extracted. Grepping the app confirmed no formula feature was requested anywhere, so equations arrived as whatever OCR guessed — usually garbled for real math.

Document Intelligence exposes a FORMULAS add-on, already present in the pinned SDK, which returns equations as LaTeX. It is billed per page, so:

  • it sits behind a new admin toggle that defaults to off
  • it is guarded by the layout-mode check, because the add-on applies to that model only
  • the admin UI states plainly that it is a billed add-on that adds per-page cost

A test asserts the default-off value, the persistence path, the cost warning in the UI, and that the feature is positioned behind the layout guard rather than applied unconditionally.

Scope note

This does not cover equations authored in modern Word, which are stored as OMML markup rather than images and so are outside embedded image analysis entirely. Legacy Equation Editor and MathType objects are stored with a metafile preview, and those are covered by the carving change above. The feature doc records both facts so the boundary is not misremembered later.

Testing

Four new tests (legacy carving, non-Office binary safety, decoy signature rejection, formula opt-in contract). The embedded image suite is 20/20; Content Understanding, legacy Word, Document Intelligence, and route policy suites all still pass.

One fixture bug worth mentioning: the synthetic EMF used in tests was ~158 bytes and was correctly rejected by the 2 KB minimum-size filter. I padded the fixture with a valid comment record to make it realistic rather than lowering the threshold, since that filter exists to drop icons and spacers.

Version bumped to 0.250.224.

Refs #1277

Two gaps surfaced while answering questions about equation and legacy
format coverage.

Legacy .doc and .ppt images. Embedded image analysis covered only DOCX
and PPTX, because legacy Office files are OLE compound documents rather
than zip packages and have no media parts to enumerate. Their pictures
and embedded equation previews are still stored as intact metafile
blobs, so they are now carved out by signature using the length recorded
in the metafile's own header, then rasterized and analyzed like any
other embedded image. Validation checks the record type, the signature
position, and that the declared length fits the remaining bytes, so a
coincidental byte sequence is not mistaken for an image. Duplicate
collapsing and the per-document cap still apply, and a file that is
neither zip nor OLE yields nothing rather than raising.

Optional formula extraction. Equations were never extracted: no formula
feature was requested anywhere, so they arrived as whatever OCR guessed.
Document Intelligence exposes a FORMULAS add-on, present in the pinned
SDK, which returns equations as LaTeX. It is billed per page, so it is
behind a new admin toggle that defaults to off, and it is guarded by the
layout-mode check because the add-on applies to that model only. The
admin UI states plainly that it is a billed add-on.

This does not cover equations authored in modern Word, which are stored
as OMML markup rather than images. Legacy Equation Editor and MathType
objects are stored with a metafile preview and are covered by the
carving change above.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@paullizer
Paul Lizer (paullizer) merged commit 39245ab into Development Aug 18, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant