Skip to content

Test: guard the repository paths README.md tells users to download - #185

Open
AmaadMartin wants to merge 2 commits into
mainfrom
fix/readme-download-contract-test
Open

Test: guard the repository paths README.md tells users to download#185
AmaadMartin wants to merge 2 commits into
mainfrom
fix/readme-download-contract-test

Conversation

@AmaadMartin

@AmaadMartin AmaadMartin commented Aug 8, 2026

Copy link
Copy Markdown
Owner

Please ensure you have read the contribution guide before creating a pull request.

Link to Issue or Description of Change

  1. Link to an existing issue (if applicable):

  2. Or, if no issue exists, describe the change:

Problem: README.md tells users to curl four paths out of this repository. Nothing asserts that those paths exist; for a while one of them did not, and the test suite stayed green through the whole window. scripts/update_constraints.sh and the pyproject.toml classifiers can also drift apart, which leaves a supported interpreter with no published constraints file.

Solution: One new test module reads the real README.md, the real scripts/update_constraints.sh, the pyproject.toml classifiers and the five committed constraints files. It fails when a documented path is absent from the checkout, when the two interpreter lists disagree, when a constraints file is missing, empty or no longer pins exactly, or when the two-line header stops carrying the --exclude-newer date that update_constraints.sh --check reads back out of it. A last test asserts the README still documents the recipe, so the guard cannot be defused by deleting its subject. No production file changes: the diff is one added file.

Collision check: I compared this against every open pull request on the fork. Four are adjacent. #108 adds a README link guard, but it also deletes the install recipe on the premise that no constraints file was ever published; that premise no longer holds, so I did not stack on it. #159 pins how the files are generated, #149 the generator CLI, #163 the pre-commit hook. None of them guards that a documented download exists.

Testing Plan

Unit Tests:

  • I have added or updated unit tests for my change.
  • All unit tests pass locally.

pytest tests/unittests/test_constraints_files.py -q gives 27 passed.

Every guard was run against a mutated tree and confirmed to fail:

Mutation Failing test
mv constraints-3.10.txt /tmp/ test_readme_repo_references_resolve[constraints-3.10.txt], test_constraints_file_exists_for_every_supported_version[3.10]
Append a raw.githubusercontent.com/.../nope.txt link to README.md test_readme_repo_references_resolve[nope.txt]
PYTHON_VERSIONS=() in the script test_supported_versions_match_pyproject_classifiers
Delete the 3.14 classifier from pyproject.toml test_supported_versions_match_pyproject_classifiers
Cut one space from the # uv pip compile header test_constraints_header_is_machine_readable[3.11]
Change one a2a-sdk==1.1.1 pin to >= test_constraints_files_pin_exact_versions[3.12]
Delete the install recipe from README.md test_readme_documents_the_constraints_install_recipe, test_readme_reference_extraction_is_not_empty
Drop blob|tree from the URL pattern, so the CI badge leaks in test_repo_paths_in_captures_only_paths_into_this_repository and 4 others

The third mutation is the one worth calling out. An empty PYTHON_VERSIONS array makes the parametrized tests collect zero cases, which pytest reports as skipped, not failed. test_supported_versions_match_pyproject_classifiers asserts the list is non-empty for exactly that reason.

Branch coverage of the new module is 95%, measured with pytest --cov --cov-branch. Two environment guards account for the shortfall. Lines 49-50 are the tomli fallback that Python 3.11+ never takes; CI runs the 3.10 leg, which does take it. Line 91 is the module-level pytest.skip for a tree that does not export the source checkout layout. I verified that path by moving scripts/update_constraints.sh aside and re-running: 1 skipped. I kept both rather than delete a guard to raise the number.

The module takes the repository root from the directory's shared isolated_import_utils.REPO_ROOT.

Manual End-to-End (E2E) Tests:

curl -fsS -o /tmp/constraints-3.10.txt \
  https://raw.githubusercontent.com/google/adk-python/main/constraints-3.10.txt
head -2 /tmp/constraints-3.10.txt
wc -l /tmp/constraints-3.10.txt
diff /tmp/constraints-3.10.txt constraints-3.10.txt

The download returns the two-line uv header, 1951 lines, and is identical to the committed file.

pytest tests/unittests --collect-only -q -k constraints_files

Collects the 29 new tests from the suite root, so the CI unit-test job runs them.

CI: Unit Tests, Mypy Check and A2A v0.3 Tests pass on every interpreter. Pre-commit Linter fails, and it fails for a reason this change does not touch: the update-constraints hook regenerates the files against a rolling "4 days ago" date and rewrites the --exclude-newer header, so it exits 1. The same job is red on main at 352d11d, the commit this branch sits on, and on the four main runs before it.

Checklist

  • I have read the CONTRIBUTING.md document.
  • I have performed a self-review of my own code.
  • I have commented my code, particularly in hard-to-understand areas.
  • I have added tests that prove my fix is effective or that my feature works.
  • New and existing unit tests pass locally with my changes.

Amaad Martin added 2 commits August 8, 2026 10:25
README.md publishes URLs into this repository -- the logo, docs/guides,
contributing/samples and the constraints-<ver>.txt the install recipe
tells users to curl. Nothing asserted those paths exist, so the window
in which no constraints file was committed passed with a green suite.

The new module also pins the parts of the download contract that make
the file worth downloading: every interpreter in the generator script's
PYTHON_VERSIONS array has a non-empty constraints file, that list agrees
with the pyproject classifiers, each file keeps the two-line header that
update_constraints.sh --check parses its snapshot date from, and every
requirement line pins exactly.
…constant

The module hand-rolled its own ancestor walk when tests/unittests already
exports REPO_ROOT. Reuse it, and drop the two meta-tests of the deleted
helper: the module-level skip guarantees the condition the first one
asserted, so it could never fail.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant