perf(semseg): in-process numpy mask fast path (response_mask_format)

[theo-roboflow]

Motivation

Stacked on #2728. In-process, the semantic segmentation model PNG-encodes + base64s two full-resolution masks only for the workflows block to immediately base64-decode + PNG-decode them. At 16.1 MP on a Jetson AGX Orin that round-trip measured ~512 ms/frame (PIL.save 382 ms + cv2.imdecode 121 ms + base64/framing) while the model forward was 2.8 ms. Base64 itself and the GPU→CPU copy are nearly free — it is PNG compression and decompression that costs.

What changed

SemanticSegmentationInferenceRequest gains response_mask_format: "base64_png" (default) | "numpy", following the existing response_mask_format precedent on the instance segmentation request. With "numpy", the model puts the raw uint8 arrays in the mask fields and the workflow block (v1 + v2, which now request it in run_locally) consumes them directly — no PNG anywhere on the in-process path. Implemented for both the inference-models adapter and the legacy ONNX semantic segmentation base.

Wire-contract safety

"numpy" is an in-process contract; every serialization boundary still produces base64 PNG strings:

• HTTP (process_inference_request) and the enterprise parallel postprocess task coerce "numpy" → "base64_png" before inference. Both paths serialize via python-mode model_dump + orjson/stdlib-json, where pydantic field serializers do not run — without the coercion an opted-in HTTP caller would have received nested int arrays (orjson OPT_SERIALIZE_NUMPY). Found by adversarial review, fixed at the boundary.
• The mask fields keep a string JSON schema (json_schema_extra pins type: string) and a when_used="json" field serializer lazily encodes ndarray values to base64 PNG for any json-mode serialization (model_dump_json, FastAPI response_model), so even a "numpy" response that reaches JSON is encoded correctly.
• Active learning only registers on the HTTP path (active_learning_eligible), which is coerced above — it never sees ndarray masks. Workflow blocks call infer_from_request_sync without that flag.
• run_remotely is untouched: the SDK client doesn't send the field, servers default to base64.

Trade-off note: the mask fields' python type widens from str to Any (schema unchanged). In-process consumers that python-dump a "numpy" response and hand it to a serializer that can't handle ndarrays would need the json-mode dump — only relevant to callers who explicitly opt in.

Measured (on-device, 16.1 MP, Jetson AGX Orin, 600+ frames converged)

Validated via a byte-identical custom_modules shim in roboflow-edge (branch claude/semseg-perf-fixes-4aa779) before porting here. Full stack (this PR + #2728):

configuration	wall/frame	plain fps
baseline (inference 1.3.7)	945.6 ms	1.034
both PRs	224.3 ms	4.48–4.88 (mean ~4.7)

png_b64_encode disappears entirely from the per-stage profile; sv_convert drops from 500 ms to 140 ms (remaining: RLE encode 40 ms + one F-order conversion + per-class ops).

Correctness

• Workflow outputs identical between formats: new tests assert the numpy path produces exactly the base64 path's RLE counts, xyxy, class ids, confidences (test_v1.py/test_v2.py).
• Entity round-trip tests: python-mode dump passes ndarrays through untouched; model_dump_json produces base64 PNG that decodes back pixel-identical; string masks serialize unchanged (test_semantic_segmentation_masks.py).
• Adapter numpy-mode test verifies raw arrays match the tensors and no encode happens (test_inference_models_adapters.py).

Review round (applied)

An adversarial multi-agent review pass was run over both PRs before marking ready; fixes applied here:

• The numpy→base64_png boundary coercion now lives in one owned helper, ensure_wire_safe_mask_format (entities/requests), called by both process_inference_request and the enterprise parallel postprocess task, and it logs a warning when it downgrades an explicit request (matching the base.py dropped-confidence precedent). Unit tests cover coercion, default, and non-semseg no-op.
• The comments on both model-side numpy branches were corrected: the when_used="json" field serializer covers json-mode dumps only — the python-dump + orjson wire boundaries do NOT run it, which is exactly why the boundary coercion is load-bearing. (Without it, orjson's OPT_SERIALIZE_NUMPY would ship a leaked mask as a giant nested int array with a 200.)
• The legacy ONNX base's numpy branch gained a unit test asserting numpy/base64 output parity.

Reproduction scripts + percentile table (reviewer-requested round)

The benchmark and parity scripts behind the numbers in this stack are committed in the base PR (#2728) under development/benchmarks/semantic_segmentation/ and run unchanged on this branch, where the pr-numpy mode (this PR's fast path) activates via feature detection:

• benchmark_postprocessing.py — per-frame latency of the model-side mask encode and the block's _convert_to_sv_detections, with mean/p50/p95/p99 per stage, in baseline / pr-base64 (perf(semseg): present_class_ids hint + single F-order conversion in workflow post-processing #2728 only) / pr-numpy (this PR) modes; synthetic 16.1 MP input generator with full metadata in the report, --label-map/--confidence-map for real frames, --memory for per-mode subprocess memory measurement.
• check_output_parity.py — byte-identical output check (RLE counts/size, xyxy, class ids/names, float32-exact confidences, confidence_mask) against the pre-PR reference across transport × hint combinations — including numpy-transport vs base64-transport equality this PR guarantees — on a full 16.13 MP map, non-zero exit on mismatch. Latest run at this head: ALL PARITY CHECKS PASSED.

x86 percentile table (isolated post-processing path)

Input: synthetic 5320×3032 (16.13 MP) uint8 label map, 4 foreground classes (44.9% fg pixels), PNG+base64 payloads 227 KiB (seg) / 125 KiB (conf), seed 42. Host: macOS arm64 (Apple silicon, 18 cores), Python 3.11.14, numpy 2.3.5, OpenCV 4.10.0, Pillow 12.3.0, torch 2.13.0 (CPU). 3 warmup + 100 measured iterations per mode, tree = this head:

mode	stage	mean ms	p50 ms	p95 ms	p99 ms
baseline	model_side (PNG+b64 encode)	84.9	84.7	86.4	87.8
baseline	convert	142.4	142.2	144.1	149.3
baseline	total	227.4	227.0	229.8	234.3
pr-base64 (#2728 only)	model_side (encode + hint)	111.9	110.5	117.7	123.8
pr-base64 (#2728 only)	convert	97.4	96.6	101.7	109.3
pr-base64 (#2728 only)	total	209.3	207.3	219.7	234.7
pr-numpy (this PR)	model_side (hint only)	25.0	24.9	25.5	26.4
pr-numpy (this PR)	convert	50.4	50.4	50.9	51.9
pr-numpy (this PR)	total	75.5	75.3	76.3	78.3

Reading notes:

• The PNG round-trip this PR removes is the largest single cost in both base64 rows (~85 ms encode + ~50 ms decode inside convert on this host; 383 + 121 ms on the Orin's cores). pr-numpy's model_side is purely the present_class_ids bincount, which on this CPU-only host runs on CPU (~25 ms) but on the CUDA/TRT device runs where the tensor lives.
• Absolute numbers are host-specific; the end-to-end on-device tables (full video pipeline, Jetson AGX Orin) are in "Measured" / "Native on-device validation" above. The on-device p50/p95/p99 tables (baseline vs this stack, fresh containers, TRT-verified) are in this comment.

Open questions for maintainers

1. Should "numpy" be in the public schema at all? Today it's a public enum value that HTTP silently (but now loudly-logged) coerces. Alternatives: model it as an internal field excluded from the schema (the stream_pipeline_context_id pattern), or return 422 over HTTP. Current draft keeps it public + coerce + log; happy to switch.
2. Field naming. response_mask_format collides with the instance-seg field of the same name but a disjoint domain (polygon|rle), which inference_sdk.InferenceConfiguration also types with the instance-seg vocabulary. Also note: an out-of-domain value in a semseg body was previously ignored (pydantic default) and now 422s. Rename (mask_transport? mask_encoding?) or keep?
3. Any typing trade-off. The mask fields' python type widens str→Any (wire schema pinned to string via json_schema_extra), which drops construction-time validation; the lazy encoder also uses np.asarray(..., uint8) (silent wrap) where the eager img_to_b64_str raises on non-uint8. A field_validator accepting only str | uint8 ndarray would restore fail-fast — worth it?
4. Three PNG-encoder copies now exist (two img_to_b64_str + the entity serializer). A shared uint8_mask_to_base64_png helper would prevent byte-level drift between eager and lazy encodings.
5. TINY_CACHE=False interaction. With that non-default setting, the model-monitoring cache write runs jsonable_encoder (json mode) per frame, which re-triggers the full-resolution PNG encode the fast path removes. Default TINY_CACHE=True is unaffected. Document, or strip mask fields from semseg cache payloads?
6. Known test gaps. The HTTP coercion is unit-tested at the helper level but not via a TestClient route test, and nothing pins that response_mask_format="numpy" survives the request.dict() → postprocess(**kwargs) chain end-to-end (if a link dropped it, the block would silently fall back to base64 with byte-identical outputs — only the perf win disappears). Both are straightforward follow-ups if wanted before merge.

Native on-device validation (no shim)

To rule out shim artifacts, both PR commits were cherry-picked onto the v1.3.7 tag (branch theo/semseg-prs-on-1.3.7 — a clean pick, confirming the touched files are identical between 1.3.7 and current main) and the 9 changed files were overlaid onto the device's 1.3.7 image directly, no monkey-patching involved. Same Jetson AGX Orin, same 16.1 MP source, and the same yolo26n model version as the baseline row, on TRT from the cached engine (warmup_backends_resolved → backend: trt; the execution_environment event confirms the overlaid adapter loaded the model):

	baseline (1.3.7)	shim validation	native (this code)
wall/frame (instrumented)	945.6 ms	224.3 ms	219.7 ms
sv_convert	499.6 ms	140.4 ms	137.8 ms
png_b64_encode	390.7 ms	eliminated	eliminated
sustained inference fps	1.03	4.5–4.9	4.4–4.6

2,240 frames converged; the per-class-ops residual inside sv_convert matches the shim run to 0.1 ms (65.8 ms both). Also reproduced on a second model version (yolo26n-v4: 214.8 ms/frame).

🤖 Generated with Claude Code

[CLAassistant]

All committers have signed the CLA.

[theo-roboflow]

Memory impact review (item 4 of the reviewer checklist — analysis verified against the code, plus measurements)

What changes hands, per frame at 16.13 MP:

1. With response_mask_format="numpy" the response object holds two uint8 ndarrays (2 × 16.13 MB) instead of two base64-PNG strings (~0.6 MB combined). That is not a net new cost: the base64 path materializes the same two full-resolution arrays anyway when the block decodes them — plus intermediates the numpy path never allocates (the tensor.cpu().numpy() staging copy feeding PIL, the PNG encode buffer, the base64 string, the decode byte buffer, and cv2.imdecode's output). Comparing peak per in-flight frame, the numpy path is the smaller one — measured below.
2. Retention window: the strings die once decoded, while the ndarrays live as long as the response object. In the workflow path the response is converted and dropped within the same run_locally call, so the envelope is one frame either way. The known exception is non-default TINY_CACHE=False (model-monitoring cache re-encodes per frame) — already flagged in the open questions.
3. Wire boundaries cannot retain or serialize ndarray masks: ensure_wire_safe_mask_format coerces numpy → base64_png on the HTTP and enterprise-parallel paths before inference. Besides correctness, that is the memory-relevant guard — without it, orjson's OPT_SERIALIZE_NUMPY would expand a leaked 16 MP mask into a nested int array (~hundreds of MB of JSON) with a 200 status. Unit-tested at both boundaries.
4. GPU side is untouched by this PR (masks already came to CPU in the base64 path; the transfer itself measured ~7 ms/frame — it is PNG compression, not the copy, that this PR removes).

Measured (x86, benchmark_postprocessing.py --memory from #2728, per-mode fresh subprocesses, 5 iterations, 16.13 MP synthetic input):

mode	tracemalloc peak	RSS growth over iterations
baseline	129.9 MB	0.0 MB
pr-base64 (#2728 only)	146.1 MB	0.0 MB
pr-numpy (this PR)	113.2 MB	0.0 MB

The numpy fast path has the lowest peak (the encode/decode intermediates are gone) and, like the others, retains nothing across iterations.

On-device (Jetson AGX Orin 64 GB, container-level telemetry, same 16.1 MP stream + yolo26n-sem TRT): the image carrying both PRs (native overlay on 1.3.7) held 3.69–3.72 GB flat across ~16 h of continuous streaming, vs a 3.3–3.9 GB envelope for the baseline-image windows earlier the same week — within noise, no drift. A fresh controlled A/B runs together with the queued percentile re-measurement.

🤖 Generated with Claude Code

[theo-roboflow]

On-device percentile tables (completes item 3 of the reviewer checklist — measured 2026-07-31 on the target device)

Setup: Jetson AGX Orin 64 GB (JetPack 6.2), 5320×3032 (16.13 MP) RTSP source at ~7 fps (on-device simulator replaying real Basler area-scan footage), model wrappertestsemanticseg-5-yolo26n-sem-t1 (yolo26n semantic seg @544²) on TensorRT (confirmed via warmup_backends_resolved for each run), OMP/OPENCV_NUM_THREADS=4, solo stream, fresh inference container per configuration. Stage profiler: roboflow-edge SEMSEG_TIMING with p99 + end-to-end wall percentiles (frame-to-frame period sampled at the frame-boundary stage, 400-sample window). Baseline image = roboflow-edge theo/semseg-baseline-main @ 76b3a59 (inference 1.3.7 unmodified); PR image = theo/semseg-native-1.3.7-overlay @ 48284f1 (1.3.7 + this stack's commits overlaid natively, no monkey-patching). Real per-frame payloads in the baseline run: seg 246 KiB + conf 249 KiB PNG+b64, 3.77 foreground classes/frame, 31.9 MP of masks/frame.

End-to-end wall per frame (baseline converged at 1,020 frames; PR at 3,520):

	mean	p50	p95	p99	instrumented fps
baseline 1.3.7	945.7 ms	948.3 ms	1022.3 ms	1059.0 ms	0.97–1.07
both PRs	214.9 ms	216.7 ms	259.1 ms	269.3 ms	4.60–4.63
speedup	4.40×	4.38×	3.95×	3.93×	~4.4×

(Earlier plain-fps runs with the profiler off sustained 4.48–4.88; the tail speedup is slightly below the mean speedup because the surviving stages — GPU post-processing and RLE — carry proportionally more of their own variance once the PNG round-trip is gone.)

Per stage, ms/frame (mean | p50 / p95 / p99):

stage	baseline 1.3.7	both PRs
pre_process	4.6 | 3.9 / 4.9 / 21.7	4.7 | 4.2 / 5.3 / 23.0
model forward (TRT)	2.9 | 2.4 / 2.9 / 21.6	2.6 | 2.4 / 2.8 / 4.0
gpu_postproc (upsample+argmax)	35.1 | 32.1 / 43.0 / 73.0	34.3 | 32.0 / 54.5 / 70.5
png_b64_encode (2 calls/frame)¹	388.5 | 202.6 / 253.0 / 275.6¹	eliminated (#2729)
sv_convert	505.3 | 509.2 / 561.8 / 598.9	133.7 | 136.8 / 175.6 / 185.8
unaccounted (video decode, framework)²	9.3	39.6²

¹ per-call percentiles (two mask encodes per frame); the mean is the per-frame total. ² the PR path's tensor.cpu() + present_class_ids bincount run outside the profiled stages, so they land in "unaccounted" — expected, matches the earlier shim validation.

Inside sv_convert (the block's _convert_to_sv_detections): np.unique 181.7 → 0 (#2728 hint), cv2.imdecode 121.0 → 0 and b64decode 0.5 → 0 (#2729 numpy transport), np.asfortranarray 105.9 → 29.2 (single F-order conversion, #2728), rle.encode 39.4 (unchanged, contract-bound), residual per-class ops ~55→65 (the two cheap == scans of the dual-mask design).

Memory (same hour, fresh containers, identical model/stream): inference-container steady state 3244–3344 MB (baseline window) vs 3133–3205 MB (PR window) — the PR image sits slightly lower while sustaining 4.4× the frame rate, consistent with the analysis in the memory-impact comment above (encode/decode intermediates gone).

Raw profiler log blocks and the exact measurement runbook are archived with the branch (roboflow-edge theo/semseg-baseline-main @ 76b3a59, theo/semseg-native-1.3.7-overlay @ 48284f1 — both carry the p99 profiler commit).

🤖 Generated with Claude Code

[theo-roboflow]

Orin-native Nsight Systems traces (profiling-skill follow-up — same snippets, now captured on the target device)

Both profiling snippets (inference_profiling/snippets/semseg_postproc_{baseline,pr}, built per development/profiling/skills/snippet_extraction/SKILL.md) were re-run on the Jetson AGX Orin itself with --device cuda: nsys 2024.5.4 (JetPack 6.2 tegra build) inside an ephemeral container from the same image as the deployed overlay, canonical --capture-range=nvtx flags from --print-nsys-command, CPU sampling on, 2 warmup + 10 measured iterations, 16.13 MP synthetic input from the committed generator. NVTX medians:

range	baseline	PR stack
iteration (end-to-end)	~991 ms	~186 ms
png_b64_encode	444.0 ms	— (stage gone)
convert_to_sv_detections	546.9 ms	154.8 ms
├ decode_masks (+conf)	123.9 ms	~0 (ndarray pass-through)
├ class_scan_np_unique → hint_or_unique	188.9 ms	~0 (hint honored)
├ per_class_bbox_conf	56.7 ms	72.4 ms¹
├ rle_encode	175.6 ms (per-class asfortranarray)	52.8 ms (F-order no-op)
├ asfortranarray_once	—	26.4 ms
present_class_ids_hint (GPU)	—	28.0 ms²

¹ the dual-mask design deliberately pays two cheap C-order == scans here to keep bbox/conf reductions off F-order layouts (the earlier variant that avoided this was net-negative on Orin). ² the hint runs on the GPU where the label map lives — the trace shows torch's CUB kernels (cub::DeviceSelect::Flagged, cub::DeviceReduce::Sum) inside this range, and the baseline trace's CUDA-kernel summary is literally empty (its post-processing never touches the GPU). Traces were captured while the production stream kept running on the same device (deliberate — it shows the code under real contention), so absolute numbers sit slightly above the solo-stream pipeline tables; the structure and ordering match them exactly.

Artifacts: trace.nsys-rep per snippet + nvtx_sum/cuda_gpu_sum CSVs (shared alongside the x86 captures in the Slack thread; regenerate any time with the snippet READMEs' commands — on-device runbook: clone the branch, copy the two snippet dirs, run nsys from the L4T r36.4 repo inside a container off the deployment image).

🤖 Generated with Claude Code