Summary
_checkpoint_path(pipeline_dir, project_id, stage) computes pipeline_dir / project_id / f"checkpoint_{stage}.json" with zero validation. write_checkpoint() then does path.parent.mkdir(parents=True, exist_ok=True) unconditionally. Neither write_checkpoint's one-line docstring, _checkpoint_path's (nonexistent) docstring, nor AGENT_GUIDE.md (which never uses the identifier pipeline_dir at all) states the expected semantics: that pipeline_dir is meant to be the projects root, and the project's own directory is pipeline_dir / project_id.
Real-world impact
Passed a project's own directory as pipeline_dir (instead of its parent) while writing a bridge script this session — silently created a wrong, doubly-nested directory tree instead of failing loudly. Every current production caller I could find passes a fixed root constant, so this hasn't caused an observed defect in shipped code, but it's a latent footgun any new script is one typo away from hitting.
Why this isn't a hard-validation fix
I initially drafted a fix that raises if pipeline_dir/project_id/project.json doesn't exist — but running the existing test suite immediately surfaced 13 failures: tests intentionally call write_checkpoint() directly on fresh tmp_path fixtures with no prior init_project() call, relying on write_checkpoint to bootstrap the directory on demand. A missing marker file is genuinely indistinguishable from a legitimate first-time bootstrap, so a hard validation check isn't safe here.
Proposed fix
Docs-only: expand write_checkpoint's docstring to state the path contract precisely (which directory pipeline_dir should be, what happens if you pass the wrong one), and add the same note to AGENT_GUIDE.md near the existing checkpoint-path convention (~line 608).
A PR implementing this will follow.
Summary
_checkpoint_path(pipeline_dir, project_id, stage)computespipeline_dir / project_id / f"checkpoint_{stage}.json"with zero validation.write_checkpoint()then doespath.parent.mkdir(parents=True, exist_ok=True)unconditionally. Neitherwrite_checkpoint's one-line docstring,_checkpoint_path's (nonexistent) docstring, norAGENT_GUIDE.md(which never uses the identifierpipeline_dirat all) states the expected semantics: thatpipeline_diris meant to be the projects root, and the project's own directory ispipeline_dir / project_id.Real-world impact
Passed a project's own directory as
pipeline_dir(instead of its parent) while writing a bridge script this session — silently created a wrong, doubly-nested directory tree instead of failing loudly. Every current production caller I could find passes a fixed root constant, so this hasn't caused an observed defect in shipped code, but it's a latent footgun any new script is one typo away from hitting.Why this isn't a hard-validation fix
I initially drafted a fix that raises if
pipeline_dir/project_id/project.jsondoesn't exist — but running the existing test suite immediately surfaced 13 failures: tests intentionally callwrite_checkpoint()directly on freshtmp_pathfixtures with no priorinit_project()call, relying onwrite_checkpointto bootstrap the directory on demand. A missing marker file is genuinely indistinguishable from a legitimate first-time bootstrap, so a hard validation check isn't safe here.Proposed fix
Docs-only: expand
write_checkpoint's docstring to state the path contract precisely (which directorypipeline_dirshould be, what happens if you pass the wrong one), and add the same note toAGENT_GUIDE.mdnear the existing checkpoint-path convention (~line 608).A PR implementing this will follow.