Skip to content

environments setup-local: consolidate databricks-connect to a single managed pin - #6255

Open
rugpanov wants to merge 2 commits into
mainfrom
setup-local-dbconnect-reconcile
Open

environments setup-local: consolidate databricks-connect to a single managed pin#6255
rugpanov wants to merge 2 commits into
mainfrom
setup-local-dbconnect-reconcile

Conversation

@rugpanov

@rugpanov rugpanov commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Problem

databricks environments setup-local managed databricks-connect only inside [dependency-groups].dev. When a project (e.g. one generated from the data-engineering template) also pins databricks-connect in [project].dependencies, setup-local set the dev pin to the compute-matched version but left the [project].dependencies pin untouched. uv then had to co-install two disjoint requirements (==15.1.* and >=16.0,<17) and failed with a cryptic unsatisfiable-resolution dump at the provision phase. A warning meant to catch this never fired (it couldn't parse ==X.* wildcards), and warnings are advisory anyway.

Change

In the install flow, databricks-connect is now fully owned by setup-local:

  • After the managed pin lands in [dependency-groups].dev, a new consolidation pass (removeStrayDatabricksConnect) removes every other databricks-connect pin — from [project].dependencies, each [project.optional-dependencies] extra, and every dependency group — so exactly one requirement survives. Each removed pin is surfaced with the new W_DBCONNECT_CONSOLIDATED warning (naming the location).
  • --constraints-only mode leaves databricks-connect completely untouched.
  • parseClause now models PEP 440 ==X.* prefix-match wildcards, so a conflict between a wildcard user pin and the environment's constraint-dependencies is reported (W_DBCONNECT_PIN_DUPLICATED / W_USER_CONSTRAINT_CONFLICT) rather than silently missed.

The consolidation preserves the file's other bytes: a comment on a surviving element (or the array's opening line) is kept when an adjacent databricks-connect element is removed. The line-based removal matches the same shapes the rewrite does (double-quoted elements under bare-key arrays). Rarer valid spellings it doesn't reach — single-quoted pins, quoted TOML keys, inline-table forms — are left in place but still surfaced as W_DBCONNECT_PIN_DUPLICATED when their range is disjoint from the managed pin.

No changelog fragment: environments setup-local is not yet released (its own .nextchanges fragment is still pending), so this fix ships in the same release that introduces the command.

Testing

  • Unit tests in libs/localenv/merge_test.go and warnings_test.go (consolidation across all three location types, empty-array/trailing-comma handling, comment-preceded strays with comment preservation, idempotency, constraints-only no-op, ==X.* wildcard disjoint math).
  • Acceptance tests: updated merge-warnings / merge-warnings-json; added dbconnect-consolidate-check, constraints-only-existing, and wildcard-constraint-conflict-check.
  • Verified end-to-end against the original repro: uv sync now resolves cleanly (the pre-existing conflict is gone).

This pull request and its description were written by Isaac.

…managed pin

setup-local managed databricks-connect only in [dependency-groups].dev, so a
databricks-connect requirement shipped by a template in [project].dependencies
conflicted with the managed dev pin and left the project unresolvable — `uv sync`
failed with an unsatisfiable-resolution error.

In the install flow, databricks-connect is now fully owned by setup-local: after
the managed pin lands in the dev group, every other databricks-connect pin is
removed from [project].dependencies, each [project.optional-dependencies] extra,
and every dependency group, so exactly one requirement survives. Each removed pin
is surfaced with the new W_DBCONNECT_CONSOLIDATED warning. --constraints-only
leaves databricks-connect untouched. parseClause now also models ==X.* prefix-match
wildcards so a conflict between a wildcard user pin and the environment's
constraint-dependencies is reported instead of silently missed.

Co-authored-by: Isaac
@github-actions

github-actions Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Approval status: pending

/acceptance/localenv/ - needs approval

14 files changed
Eligible: @rclarey, @anton-107, @misha-db

/libs/localenv/ - needs approval

6 files changed
Eligible: @rclarey, @anton-107, @misha-db

Any maintainer (@andrewnester, @anton-107, @denik, @pietern, @shreyas-goenka, @simonfaltum, @renaudhartert-db, @janniklasrose, @lennartkats-db, @rclarey) can approve all areas.
See OWNERS for ownership rules.

…log fragment

Code review found that consolidation could silently delete an inline comment that
belonged to a surviving element: splitTopLevelElements groups the text after a
comma into the next element's token, so a trailing comment on the element before a
removed databricks-connect pin was dropped with it, violating MergeManaged's
comment-preservation contract. removeDbconnectFromArraySpan now carries those
leading comment lines onto the next retained token so they stay on their line.

Also drop the .nextchanges fragment: environments setup-local is still unreleased
(its own fragment is pending), so this fix lands in the same release that
introduces the command — a separate changelog entry would be redundant.

Co-authored-by: Isaac
@rugpanov
rugpanov force-pushed the setup-local-dbconnect-reconcile branch from 403aed4 to d528701 Compare August 12, 2026 14:45
@eng-dev-ecosystem-bot

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: d528701

Run: 31608672636

Env 💚​RECOVERED 🙈​SKIP ✅​pass 🙈​skip Time
💚​ aws linux 4 4 286 1139 10:01
💚​ aws windows 4 4 288 1137 7:56
💚​ azure linux 4 4 285 1139 9:29
💚​ azure windows 4 4 287 1137 7:51
💚​ gcp linux 1 5 286 1139 9:14
💚​ gcp windows 1 5 288 1137 7:17
8 interesting tests: 4 RECOVERED, 4 SKIP
Test Name aws linux aws windows azure linux azure windows gcp linux gcp windows
💚​ TestAccept 💚​R 💚​R 💚​R 💚​R 💚​R 💚​R
🙈​ TestAccept/bundle/invariant/no_drift 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_endpoints/drift/recreated_same_name 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/bundle/resources/vector_search_indexes/recreate/embedding_dimension 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
🙈​ TestAccept/ssh/connection 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo 💚​R 💚​R 💚​R 💚​R 🙈​S 🙈​S
💚​ TestFetchRepositoryInfoAPI_FromRepo/root 💚​R 💚​R 💚​R 💚​R
💚​ TestFetchRepositoryInfoAPI_FromRepo/subdir 💚​R 💚​R 💚​R 💚​R
Top 8 slowest tests (at least 2 minutes):
duration env testname
3:15 gcp windows TestFilerRecursiveDelete/workspace_files_extensions
3:00 aws windows TestFilerReadWrite/workspace_files
2:56 azure windows TestAccept
2:56 gcp windows TestAccept
2:49 aws windows TestAccept
2:43 aws windows TestFilerRecursiveDelete/workspace_files
2:36 azure linux TestExportDir
2:12 azure windows TestSyncIncrementalFileSync

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants