fix: normalize unsupported images before sending to vision providers

[unknowbug]

Fixes #9771

Summary

OpenAI-compatible vision providers such as DeepSeek reject image formats outside webp/png/jpeg/gif with a 400 unsupported image error. AstrBot currently:

• preserves the original unsupported image_url block when local image materialization fails (return resolved_part or part);
• injects tool-returned cached images using their raw MIME type (including image/svg+xml, image/bmp, etc.) without any provider-safe conversion;
• does not recognize "unsupported image" in _handle_api_error, so a transient bad image leaves the whole agent in ERROR instead of falling back to text-only.

This PR normalizes unsupported image formats before they are sent to the provider, guards the tool cached-image review path, and adds the missing error fallback.

Changes

• media_utils: add normalize_image_for_provider():
  • keeps webp/png/jpeg/gif as-is;
  • corrects mislabeled MIME when the actual bytes are already provider-safe;
  • converts BMP/TIFF/AVIF etc. to JPEG (or PNG when transparency is present);
  • returns None for images that cannot be parsed (e.g. SVG) so callers can omit them.
• openai_source:
  • _resolve_image_part() now normalizes image data before constructing image_url;
  • _transform_content_part() replaces an unusable image with a text placeholder instead of keeping the bad image_url;
  • _is_invalid_attachment_error() now matches "unsupported image", enabling the existing text-only retry fallback.
• tool_loop_agent_runner:
  • cached tool images are normalized before being added to the LLM review context;
  • unsupported/corrupt images are skipped (the textual [Image from tool ...] marker remains).
• Tests:
  • test_media_utils.py: normalize_image_for_provider BMP conversion, unsupported SVG handling, supported MIME passthrough.
  • test_openai_source.py: unsupported-image API error fallback, BMP to JPEG resolution, SVG rejection, unresolvable context image placeholder replacement.

Notes

tool_image_cache is intentionally left unchanged: cached images may still need to be delivered to users in their original format (e.g. SVG). The provider-review boundary is now guarded by normalize_image_for_provider().

Summary by Sourcery

Ensure vision-provider requests contain only valid, supported image formats while degrading gracefully when images cannot be used.

Bug Fixes:

• Normalize unsupported or mislabeled images before sending them to vision providers, converting compatible formats and omitting unparseable images.
• Fallback to text-only requests when providers reject uploaded images as unsupported.
• Replace unresolved context images with a text placeholder and skip unusable cached tool images during provider review.

Enhancements:

• Centralize provider-safe image validation and conversion for supported MIME types, transparency, and invalid image payloads.

Tests:

• Add coverage for image normalization, format conversion, invalid image handling, placeholder replacement, cached-image filtering, and unsupported-image fallback.

[sourcery-ai]

Hey - I've found 1 issue

Prompt for AI Agents

Please address the comments from this code review:

## Individual Comments

### Comment 1
<location path="astrbot/core/utils/media_utils.py" line_range="164" />
<code_context>
+
+    supported = supported_mimes or IMAGE_PROVIDER_SUPPORTED_MIME_TYPES
+    mime = (getattr(image_data, "mime_type", "") or "").lower()
+    if mime in supported:
+        return image_data
+
</code_context>
<issue_to_address>
**issue (bug_risk):** Images whose declared MIME type is already in the provider-supported set are returned without decoding or validating their bytes. A corrupt payload or an unsupported image mislabeled as `image/png`, `image/jpeg`, `image/webp`, or `image/gif` is therefore still sent to the provider and triggers the same unsupported/invalid-image request failure this normalization is intended to prevent.

**Triggers:** When cached tool data or an image reference carries a provider-safe MIME label but contains invalid or differently formatted bytes.

**Suggested fix:** Validate the payload even for supported MIME types, and return the original data only when the detected format matches a supported provider format; otherwise convert it or return `None`.
</issue_to_address>

Sourcery assessment

Approval pending. 1 finding to address first.

Blocking findings: astrbot/core/utils/media_utils.py:164

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}