Test: Convert the reflect-and-retry plugin tests to pytest style - #246
Open
AmaadMartin wants to merge 2 commits into
Open
Test: Convert the reflect-and-retry plugin tests to pytest style#246AmaadMartin wants to merge 2 commits into
AmaadMartin wants to merge 2 commits into
Conversation
added 2 commits
August 11, 2026 23:15
…tyle The two reflect-and-retry test files were the last IsolatedAsyncioTestCase classes under tests/unittests/plugins. Drop the base class, mark all 42 async tests with @pytest.mark.asyncio, and rewrite the 122 self.assert* calls as bare asserts. The 8 assertRaises sites become pytest.raises with a match= pattern taken from the plugin source. The two dict-error sites keep Exception, because _ensure_exception wraps a non-Exception error in the base class; they now also pin type(exc) is Exception, which subsumes the old assertNotIsInstance(TypeError) regression guard. No production source is touched.
…hronous 10 of the 42 converted tests contain no await. They only build plugins and call adk_handle_model_error, _check_for_model_error and _get_model_name_from_context, which are all plain def. They were async only because IsolatedAsyncioTestCase made that the idiom, so the coroutine and its event loop bought nothing. Make those 10 plain def and drop their now-pointless asyncio marker. The collected count per file is unchanged at 18 and 24; the remaining 32 async tests keep their marker. Constructing a plugin outside a running loop is safe on the repository's Python 3.10 floor, where asyncio.Lock binds to a loop on first acquire rather than at construction. Also tighten one assertion on a line this branch already rewrites: throw_exception_if_retry_exceeded is False, not merely "is not True", which would also pass for None or 0.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Please ensure you have read the contribution guide before creating a pull request.
Link to Issue or Description of Change
N/A
Problem: The two reflect-and-retry test files were the last
unittest.IsolatedAsyncioTestCaseclasses undertests/unittests/plugins/, so the directory used two testing idioms. The guidelines require pytest andpytest.raises(SpecificError, match=...). ThreeassertRaisessites asserted only the exception type, and two of them used a bareException.Solution: I dropped the base class, marked the 32 remaining async tests with
@pytest.mark.asyncio, and rewrote the 122self.assert*calls as bare asserts. Each of the 8pytest.raisescalls now carries amatch=pattern copied from the plugin source. No production source changes.10 tests are now synchronous. They never
await. They only build plugins and calladk_handle_model_error,_check_for_model_errorand_get_model_name_from_context, which are all plaindef(_reflect_retry_model_plugin.py:79,103,120). They wereasyncbecauseIsolatedAsyncioTestCasemade that the idiom, so the coroutine and its event loop bought nothing. The collected count per file is unchanged.Collision check:
gh pr list --repo AmaadMartin/adk-python --state open --limit 100returned no pull request that touches either file. The nearest neighbours are #179 and #180 (async marker hygiene) and #168 (unused imports); I checked their file lists withgh pr diff --name-onlyand none includestest_reflect_retry_model_plugin.pyortest_reflect_retry_tool_plugin.py. This branch is based on the currentmain.The two
pytest.raises(Exception)sites are deliberate._ensure_exception(reflect_retry_tool_plugin.py:269) returnserror if isinstance(error, Exception) else Exception(str(error)), so the raised object is exactlyException, not a subclass. Both tests drive a plugin whoseextract_error_from_resultreturns a dict. Each site addsassert type(exc_info.value) is Exception, which pins the exact type and subsumes the oldassertNotIsInstance(cm.exception, TypeError)guard. Giving the plugin a specific error type is a behaviour change and is out of scope.Why dropping
IsolatedAsyncioTestCaseis safe. I deleted a comment that justified the base class, so here is why its hazard no longer applies. An earlier form of that comment cited pytest-dev/pytest-asyncio#1039. That issue is a pytest-asyncio 0.25.1 regression on Python 3.9 only; the reporter states that 3.10 through 3.13 are unaffected, and the maintainers closed it as not planned. This repository requires Python 3.10 or newer (pyproject.toml:15) and CI tests 3.10 through 3.14, so no supported version can hit it.Four further reasons:
setUp,asyncSetUp,tearDown,setUpClass, or any class attribute. Every plugin, mock and runner is built inside the test body, so nothing outlives one test's loop.ScopedFailureTracker.__init__creates anasyncio.Lock(). Since Python 3.10, aLockbinds to the running loop on first acquire, not at construction. That is also what makes the 10 synchronous tests above safe.pyproject.toml:341already setsasyncio_default_fixture_loop_scope = "function", and no async fixture is involved, so each test gets a fresh loop.testing_utils.TestInMemoryRunneralready runs under plain pytest-asyncio elsewhere, for exampletests/unittests/flows/llm_flows/test_functions_simple.py.Testing Plan
Please describe the tests that you ran to verify your changes. This is required for all PRs that are not small documentation or typo fixes.
Unit Tests:
[x] I have added or updated unit tests for my change.
[x] All unit tests pass locally.
Collected counts are unchanged: 18 for the model file and 24 for the tool file, before and after. The directory total stays 658.
The strict-mode directory run fails for a reason this change does not touch. Five other files in the directory hold unmarked async tests. I captured the 137
FAILED/ERRORnode ids before and after;diffreports them identical, and none belongs to the two converted files.Mutation testing. I ran each converted test against broken source and confirmed it fails.
"Agent model not found."->"Agent model missing."Expected regex: 'Agent model not found'{self.max_retries}Expected regex: 'failed consecutively 1 times ...'_ensure_exceptionreturnserrorunwrappedmax_retriesguard removedDID NOT RAISE ValueErrortest_plugin_initialization_default@pytest.mark.asyncioremovedasync def functions are not natively supportedRows 1, 4 and 5 target tests that are now synchronous, so the conversion did not weaken them. The last row shows the strict-mode gate reports a missing marker as a failure, not a silent skip.
Manual End-to-End (E2E) Tests:
Not applicable. This change alters no runtime behaviour.
test_hallucinating_tool_namedrives a realLlmAgentthroughtesting_utils.TestInMemoryRunnerand is carried across unchanged apart from the marker and one assertion.CI. Unit Tests pass on Python 3.10, 3.11, 3.12, 3.13 and 3.14. Mypy Check and the A2A v0.3 Tests pass on every version.
Pre-commit Linter fails, for a reason this change does not cause. The failing hook is
update-constraints, which reports "files were modified by this hook". That hook runs only onpyproject.tomlandconstraints-*.txt; this branch touches neither. The same job fails on the other open pull requests, for example #243, #244 and #245.I also ran the formatters locally on the pushed commit:
ruff check tests/unittests/plugins/reports 4 pre-existingF401errors intest_global_instruction_plugin.pyandtest_multimodal_tool_results_plugin.py. Those files are not in this change.Checklist
[x] I have read the CONTRIBUTING.md document.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have added tests that prove my fix is effective or that my feature works.
[x] New and existing unit tests pass locally with my changes.