Skip to content

fix(workflow-operator): validate zero-shot-image-classification labels before running - #7585

Open
PG1204 wants to merge 3 commits into
apache:mainfrom
PG1204:fix/hf-zero-shot-image-labels
Open

fix(workflow-operator): validate zero-shot-image-classification labels before running#7585
PG1204 wants to merge 3 commits into
apache:mainfrom
PG1204:fix/hf-zero-shot-image-labels

Conversation

@PG1204

@PG1204 PG1204 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this PR?

When the zero-shot-image-classification task ran without at least 2 candidate labels, the operator raised the error from inside the per-row loop, so it crashed mid-run instead of failing cleanly. This moves the check up front, alongside the operator's other config validations, so it fails fast with a clear message before any rows are processed.

Labels now come from the Candidate Labels field only (matching the text zero-shot-classification task); the old fallback that read labels from the prompt column is removed.

Any related issues?

Closes #7199 (the Part B follow-up to #7297, which addressed the same issue).

How was this PR tested?

Unit tests + the generated-Python compile check, plus a quick headless run of the generated operator: with 0 labels it now raises a clear "requires at least 2 Candidate Labels" error before the loop; with 2+ labels it proceeds normally.

Was this PR authored or co-authored using generative AI tooling?

Yes, this PR was co-authored with Claude in compliance with ASF policy.

@PG1204

PG1204 commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

/request-review @xuang7

@github-actions

Copy link
Copy Markdown
Contributor

Backport auto-label report

This fix: PR was checked against each actively-supported release branch. release/* labels drive the post-merge backport, so add or remove one to change where this fix lands.

Release branch Analysis
⚠️ release/v1.2 Not labeled automatically — none of the files this PR modifies exist on this branch (common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/HuggingFaceCodegenBase.scala, common/workflow-operator/src/main/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegen.scala, common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/HuggingFaceInferenceOpDescSpec.scala, common/workflow-operator/src/test/scala/org/apache/texera/amber/operator/huggingFace/codegen/ImageTaskCodegenSpec.scala). The fix may target code that isn't on this release, or the files were moved/renamed after the branch was cut. Please check and add release/v1.2 by hand if this fix should be backported here.

Auto-label run.

@github-actions

Copy link
Copy Markdown
Contributor

Automated Reviewer Suggestions

Based on the git blame history of the changed files, we recommend the following reviewers:

  • No candidates found from git blame history.

@github-actions
github-actions Bot requested a review from xuang7 August 12, 2026 04:30
@codecov-commenter

codecov-commenter commented Aug 12, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.66%. Comparing base (12169c2) to head (2c323b4).

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #7585   +/-   ##
=========================================
  Coverage     89.65%   89.66%           
- Complexity     4397     4401    +4     
=========================================
  Files          1177     1177           
  Lines         46996    46990    -6     
  Branches       5268     5268           
=========================================
- Hits          42136    42135    -1     
  Misses         3094     3094           
+ Partials       1766     1761    -5     
Flag Coverage Δ *Carryforward flag
access-control-service 70.00% <ø> (ø)
agent-service 98.62% <ø> (-0.01%) ⬇️ Carriedforward from a09260a
amber 86.38% <ø> (+0.02%) ⬆️ Carriedforward from a09260a
computing-unit-managing-service 72.46% <ø> (ø)
config-service 77.31% <ø> (ø)
file-service 68.90% <ø> (ø)
frontend 90.67% <ø> (ø) Carriedforward from a09260a
notebook-migration-service 78.89% <ø> (ø)
pyamber 97.57% <ø> (ø) Carriedforward from a09260a
workflow-compiling-service 57.89% <ø> (ø)

*This pull request uses carry forward flags. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

⚠️ Benchmark changes need a look

🟢 2 better · 🔴 7 worse · ⚪ 6 noise (<±5%) · 0 without baseline

Compared against main 12169c2 benchmarked on this same runner, so the delta is largely free of cross-runner hardware noise. The "7d avg" column still reflects the gh-pages dashboard. Treat <±5% as noise unless repeated.

Dashboard · Run

config throughput MB/s latency max Δ latest / 7d
🔴 bs=10 sw=10 sl=64 366 0.223 25,411/42,565/42,565 us 🔴 +17.3% / 🔴 +160.4%
🟢 bs=100 sw=10 sl=64 808 0.493 122,348/135,227/135,227 us 🟢 -15.2% / 🔴 +25.7%
🔴 bs=1000 sw=10 sl=64 929 0.567 1,066,366/1,204,723/1,204,723 us 🔴 +5.8% / 🔴 +16.2%
Baseline details

Latest main 12169c2 from same runner

config metric PR latest main 7d avg Δ latest Δ 7d
bs=10 sw=10 sl=64 throughput 366 tuples/sec 436 tuples/sec 774.89 tuples/sec -16.1% -52.8%
bs=10 sw=10 sl=64 MB/s 0.223 MB/s 0.266 MB/s 0.473 MB/s -16.2% -52.8%
bs=10 sw=10 sl=64 p50 25,411 us 21,655 us 12,738 us +17.3% +99.5%
bs=10 sw=10 sl=64 p95 42,565 us 36,855 us 16,348 us +15.5% +160.4%
bs=10 sw=10 sl=64 p99 42,565 us 36,855 us 18,848 us +15.5% +125.8%
bs=100 sw=10 sl=64 throughput 808 tuples/sec 804 tuples/sec 1,005 tuples/sec +0.5% -19.6%
bs=100 sw=10 sl=64 MB/s 0.493 MB/s 0.491 MB/s 0.613 MB/s +0.4% -19.6%
bs=100 sw=10 sl=64 p50 122,348 us 119,228 us 100,970 us +2.6% +21.2%
bs=100 sw=10 sl=64 p95 135,227 us 159,416 us 107,605 us -15.2% +25.7%
bs=100 sw=10 sl=64 p99 135,227 us 159,416 us 116,429 us -15.2% +16.1%
bs=1000 sw=10 sl=64 throughput 929 tuples/sec 928 tuples/sec 1,030 tuples/sec +0.1% -9.8%
bs=1000 sw=10 sl=64 MB/s 0.567 MB/s 0.566 MB/s 0.629 MB/s +0.2% -9.8%
bs=1000 sw=10 sl=64 p50 1,066,366 us 1,071,717 us 991,433 us -0.5% +7.6%
bs=1000 sw=10 sl=64 p95 1,204,723 us 1,138,683 us 1,036,668 us +5.8% +16.2%
bs=1000 sw=10 sl=64 p99 1,204,723 us 1,138,683 us 1,070,470 us +5.8% +12.5%
Raw CSV
config_idx,batch_size,schema_width,string_len,num_batches,total_ms,total_tuples,total_bytes,tuples_per_sec,mb_per_sec,lat_p50_us,lat_p95_us,lat_p99_us
0,10,10,64,20,546.88,200,128000,366,0.223,25411.09,42565.07,42565.07
1,100,10,64,20,2475.64,2000,1280000,808,0.493,122348.44,135226.63,135226.63
2,1000,10,64,20,21525.19,20000,12800000,929,0.567,1066366.15,1204723.11,1204723.11

@xuang7 xuang7 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. One thing worth a quick ack since it goes beyond what #7199 asked for: this also drops the prompt-column fallback for candidate labels, so any workflow still supplying labels through the prompt column will now fail with the config error instead. Given the fallback was a pre-#5574 transition shim and the error message points users to the right field, this seems fine.

@PG1204

PG1204 commented Aug 14, 2026

Copy link
Copy Markdown
Contributor Author

LGTM. One thing worth a quick ack since it goes beyond what #7199 asked for: this also drops the prompt-column fallback for candidate labels, so any workflow still supplying labels through the prompt column will now fail with the config error instead. Given the fallback was a pre-#5574 transition shim and the error message points users to the right field, this seems fine.

@xuang7 thanks! yep, deliberate. It makes zero-shot-image-classification consistent with text zero-shot-classification (labels come only from the candidate labels field), and the pre-loop check turns any prompt-column usage into a clear error instead of silently-wrong results. Agreed that it's fine to drop.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

HuggingFace inference operator: inconsistent input validation, asserts vs a per-row ValueError that crashes the operator

3 participants