Extract EXIF and color facts from raster image files

[lukemelia]

Now scoped to the image family only. Audio, MIDI, and the extraction-efficiency work that had accumulated here moved to #5714, which stacks on this; video is #5713 behind that.

First slice of CS-12230, covering the metadata half of CS-12237.

The gap

The image FileDefs recorded width and height and nothing else. A photograph in a realm indexed as anonymously as a generated placeholder — no camera, no capture time, no bit depth, and no way to answer "which of these images have an alpha channel".

What's here

file-formats/metadata-fields.gts — the shared shapes. Two primitives (QuantityField, CodedValueField) plus CameraCaptureField, GeoLocationField, ColorProfileField, and ExifMetadataField. One shape per metadata family rather than per extension, so "where was this taken" stays one question whether the answer came from a JPEG's GPS IFD or, later, a video container. Later slices append their own shapes here.

exif-meta-extractor.ts — a TIFF/EXIF reader using nothing but DataView and TextDecoder. The unit of work is deliberately the TIFF block, not the JPEG: EXIF is a TIFF structure several containers embed verbatim, so PNG's eXIf chunk and WebP's EXIF chunk can reuse parseExifTiffBlock as they land, and only extractExifFromJpeg knows about APP1 framing.

Color facts per container, added to the *-meta-extractor that already owns each format: PNG's IHDR, JPEG's frame header, GIF's logical screen descriptor, WebP's per-flavor alpha bit, and AVIF's pixi / colr / auxC item properties.

Two judgment calls

colorProfile is a sibling of exif, not nested inside it. The handoff nests it. EXIF's ColorSpace tag is one weak signal, while the container states bit depth, channels, and alpha outright — nesting both would mean two colorProfile blocks disagreeing about one file. The container wins, and the EXIF tag is folded in as a color-space fallback, normalized onto the same vocabulary so the persisted value doesn't depend on which container it arrived in.

Absence stays distinguishable from ignorance. A GIF's transparency is declared in a Graphic Control Extension well past the header this pass reads, so hasAlpha is left unset rather than reported false. JPEG, which has no alpha in any mode, reports false outright. Same discipline for an unrecognized code, an out-of-range coordinate, and a lone latitude with no longitude — dropped, not guessed at or clamped onto the prime meridian.

Dependency graph

The fields attach to a new RasterImageDef, not to ImageDef in card-api — which every card in every realm carries in its dependency graph. Only realms holding image files pay for the metadata modules. SvgDef stays on ImageDef: a vector has no bit depth to record. The realm indexing dependency assertions #5701 added as the guard on graph growth are unchanged and passing.

This does add one level to the image prototype chain, so image file rows gain a Raster Image entry in types and display_names. Additive — on:/type: filters against ImageDef still match.

Robustness

These parsers run during indexing against whatever bytes a realm happens to hold, so every payload read is bounds-checked and returns undefined rather than throwing. A truncated file yields partial metadata; it does not fail the extract. Directory entry counts are capped, and there's no recursion, so a malformed pointer can't loop.

Testing

• unit/image-metadata-extractor-test.ts — 44 tests / 79 assertions over hand-assembled byte fixtures: both TIFF byte orders, GPS sign and altitude reference, the EXIF 2.3 ISO tag rename, timestamp conversion with and without a recorded UTC offset, XMP-vs-EXIF APP1 discrimination, and the malformed cases (truncated heap, garbage byte-order marker, implausible entry count, illegal PNG color type).
• acceptance/image-def/jpg-image-def-test.gts — an EXIF-bearing JPEG proves a value three levels deep (exif.capture.exposureTime.displayText) survives extract → search_doc → file-meta, and that a JPEG without EXIF gains no empty exif object.
• acceptance/image-def/png-image-def-test.gts — IHDR color profile end-to-end.

ember-tsc for host, ai-bot, bot-runner, and billing; packages/base's own lint; and the full realm indexing module all pass.

The 6 authenticated images display in browser failures in the image-def suites are pre-existing — they reproduce identically on a stashed baseline, including in svg image def, which this PR doesn't touch.

One thing to know

packages/base is template-lint-only in CI (its lint script runs ember-template-lint, and that passes). The pre-commit hook additionally runs the root eslint config over staged base files, where every base .gts already fails to parse on <template>. image-file-def.gts is now the only base .gts with decorators and no template, so it parses and trips the erasable-syntax decorator rule — a rule aimed at code Node runs under --experimental-strip-types, which base card modules never are. Non-blocking and CI-green; the real fix is scoping that rule away from card realms, which belongs in a config change rather than here.

🤖 Generated with Claude Code

[github-actions]

Preview deployments

• Host staging preview

• Host production preview

Host Test Results

    1 files      1 suites   2h 50m 39s ⏱️
3 931 tests 3 917 ✅ 14 💤 0 ❌
3 950 runs  3 936 ✅ 14 💤 0 ❌

Results for commit 4ed8995.

Realm Server Test Results

    1 files      1 suites   14m 5s ⏱️
2 086 tests 2 086 ✅ 0 💤 0 ❌
2 165 runs  2 165 ✅ 0 💤 0 ❌

Results for commit 4ed8995.

[Copilot]

Pull request overview

Note

Copilot was unable to run its full agentic suite in this review.

This PR introduces a shared “raster image metadata” layer that extracts and persists EXIF and container-level pixel encoding facts (bit depth/channels/alpha/color space) across multiple image formats, with unit + acceptance coverage.

Changes:

• Add shared metadata FieldDefs (EXIF + color profile) and a RasterImageDef subclass that owns these fields.
• Implement header-level color profile extractors for PNG/JPEG/GIF/WebP/AVIF and wire them into the corresponding *Def extract paths (plus EXIF for JPEG).
• Add extensive byte-level unit tests and new acceptance tests validating end-to-end persistence into searchDoc and file-meta.

Reviewed changes

Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
packages/host/tests/unit/image-metadata-extractor-test.ts	New byte-level unit coverage for EXIF and per-format color-profile readers plus attribute assembly.
packages/host/tests/acceptance/image-def/png-image-def-test.gts	Acceptance test asserting PNG IHDR-derived color profile is persisted into searchDoc.
packages/host/tests/acceptance/image-def/jpg-image-def-test.gts	Acceptance tests asserting EXIF + JPEG color facts survive extract → index → file-meta.
packages/base/webp-meta-extractor.ts	Add WebP color profile extraction (alpha + optional ICC presence).
packages/base/webp-image-def.gts	Migrate WebP def to RasterImageDef and persist color profile attribute.
packages/base/png-meta-extractor.ts	Add PNG IHDR color profile extraction.
packages/base/png-image-def.gts	Migrate PNG def to RasterImageDef and persist color profile attribute.
packages/base/jpg-meta-extractor.ts	Add JPEG frame info + color profile extraction.
packages/base/jpg-image-def.gts	Migrate JPEG def to RasterImageDef and persist EXIF + color profile attributes.
packages/base/image-file-def.gts	Introduce RasterImageDef fields and rasterImageAttributes() assembly helper.
packages/base/image-color-profile.ts	New shared ImageColorProfile shape + pruning helper.
packages/base/gif-meta-extractor.ts	Add GIF header-derived color profile extraction.
packages/base/gif-image-def.gts	Migrate GIF def to RasterImageDef and persist color profile attribute.
packages/base/file-formats/metadata-fields.gts	New shared FieldDefs/vocabularies for EXIF + color profile UI/persistence.
packages/base/file-formats/index.ts	Export the new metadata FieldDefs via the barrel.
packages/base/exif-meta-extractor.ts	New portable EXIF parser + JPEG APP1 scanner.
packages/base/avif-meta-extractor.ts	Add AVIF color profile extraction from item properties (pixi/colr/auxC).
packages/base/avif-image-def.gts	Migrate AVIF def to RasterImageDef and persist color profile attribute.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.