CI: Streamline release pipeline and Docker image verification

[SurbhiJainUSC]

Description

This PR fixes the following issues:

• release_pipeline.yml:
  - Fix permissions for nested track_performance job (contents: write).
  - Add wait_for_tests_verification step to poll for verified pre-training and post-training images.
  - Automate PyPI release by calling pypi_release.yml upon test verification.
  - Separate TPU and GPU release candidate image build jobs.
• pypi_release.yml:
  - Add workflow_call support for automated invocation from release pipeline.
  - Remove repository_dispatch to prevent duplicate executions from individual Airflow callbacks.
  - Retain workflow_dispatch for manual/independent re-runs.
• promote_docker_image.yml:
  - Fix GITHUB_RUN_ID environment variable collision by renaming to CALLER_RUN_ID.
  - Restructure matrix to stable images and dynamically check/tag nightly images if present.
  - Standardize verified image tag to verified-e2e-${CALLER_RUN_ID}.
  - Add dynamic run-name for Airflow test callbacks.
• run_ci_tests.yml & tpu_docker_images_pipeline.yml:
  - Update CI traceability tag to verified-ci-${image_tag}.
  - Remove redundant latest tagging workarounds in CI runner.

FIXES: b/537797666

Tests

Release pipeline workflow: https://github.com/AI-Hypercomputer/maxtext/actions/runs/32514298009

Checklist

Before submitting this PR, please make sure (put X in square brackets):

• I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
• I have necessary comments in my code, particularly in hard-to-understand areas.
• I have run end-to-end tests tests and provided workload links above if applicable.
• I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

[gemini-code-assist]

Note

Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!