Add azd ai dataset extension for Foundry datasets by m7md7sien · Pull Request #9499 · Azure/azure-dev

Mohamed Hessien (m7md7sien) · 2026-08-10T00:13:43Z

Adds azd ai dataset ΓÇö a standalone extension for registering and versioning Foundry datasets.

Why its own extension

A dataset is a general Foundry asset. Fine-tuning and other scenarios need one as much as evaluation does, so the CRUD group does not belong inside the evaluation extension. This extension has its own Go module, lint workflow and release pipeline, and no dependency on azure.ai.evaluations.

The line drawn with the evaluation extension: CRUD moved, generation stayed. generate writes the datasets: entry into the evaluation configuration, so moving it would strand a config write across an extension boundary ΓÇö a generated dataset registered with the service and absent from configuration, which azd up would not reconcile. create/update/list/show/delete/versions need no configuration at all, which is why they could move.

Surface

azd ai dataset create <name>      register a dataset, publishing version 1
azd ai dataset update <name>      publish a new version
azd ai dataset list               list the project's datasets
azd ai dataset show <name>        show a version
azd ai dataset delete <name>      delete a version
azd ai dataset versions list <name>

Notable

Every user-facing string lives in one file (internal/messages/messages.go) so the whole voice of the CLI can be reviewed in one sitting. 69 messages.
TYPE, not FORMAT. The API accepts format on upload and never returns it, so the previous column was blank for every dataset ever listed.
Identity fixed. Carved from the evaluation extension, it had inherited its User-Agent (azd-ext-azure-ai-evaluations) and debug log name (azd-ai-eval-*.log), so a dataset request could not be told from an eval one in a service log or a local one.
Auth errors say what to do. A credential that cannot mint a token fails before any 401 and surfaced as a bare SDK error. It now suggests retrying first and logging in second ΓÇö observed failing transiently against a login that was valid.
A name the service will reject is refused locally. A dataset name with a space came back as a 400 wrapped in four levels of nested JSON; it is now checked against the accepted character set before the request.
A UTF-8 BOM no longer corrupts an upload. Windows editors and PowerShell write one by default, and it was being uploaded into the first row's first key rather than skipped.
Nil-guarded the endpoint resolution cascade and closed the azd client on the error paths that abandoned it.

Verified

go build, go vet -tags live,hero, go test ./... and gofmt clean. Exercised live against a Foundry project: create ΓåÆ 1.0, update ΓåÆ 2.0, versions list, show, list, and published to a local registry for a bug bash through real azd.

Not included, deliberately

No registry.json entry and no microsoft.foundry bundle dependency. Both need published artifacts; listing an unpublished extension in the bundle breaks azd extension install microsoft.foundry for everyone. Those land after the first release.

Open question for reviewers

azd ai dataset versions list <unknown-name> errors and exits 1 here, while the same command in the evaluation extension (#9500) lists nothing and exits 0. I tried making the eval side match this one and it broke two tests whose names are the argument against it ΓÇö "an unknown name lists nothing rather than failing" and "deleting an unregistered dataset is idempotent, not an error", the second because delete idempotence is verified by listing what is left.

A list is a filter rather than a lookup, so empty-and-successful is defensible and -o json callers range over the array instead of branching on an error. By that reasoning this extension's versions list is the one that should move. dataset show on an unknown name is not in question: that one is a lookup and should fail, as it does.

Draft ΓÇö not requesting reviewers yet.

Fixes #9550

Datasets are a general Foundry asset that fine-tuning and other scenarios need as much as evaluation does, so the CRUD group ships as its own extension rather than inside azure.ai.evaluations. Generation stays with the evaluation extension, because it writes that extension's config file. Standalone: no dependency on azure.ai.evaluations, its own go module, lint workflow and release pipeline.

Adds internal/messages, holding the extension's errors, warnings and status lines behind one named function each, grouped under Datasets, Config, Output and Talking to the service. Wording is unchanged; the call sites now name the situation instead of spelling out the sentence. Cobra help, wire constants and table headers stay where they are.

Three things carried over from the extension this was carved out of. The User-Agent announced azure-ai-evaluations and the debug log was written to azd-ai-eval-<date>.log, so a dataset request could not be told from an eval one in either a service log or a local one. The FORMAT column was worse than wrong: format is a field the API accepts on upload and never sends back, so the column was blank for every dataset ever listed. It shows type, which the service does return.

Carved from the same code, so it had the same two: a nil Environment or GetValue response panicking inside the endpoint cascade, and an azd client abandoned when endpoint or credential setup failed.

Same credential failure and same 401/403 path as the eval extension, which this was carved from.

Same intermittent credential failure as the eval extension.

Same three as the eval extension, which this was carved from.

azure-pipelines · 2026-08-10T00:13:56Z

Azure Pipelines: 7 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

A BOM at the head of a .jsonl is uploaded as part of the first row's first key, so every consumer of the dataset sees one malformed record -- and nothing fails until something tries to read that row. Notepad, VS Code on Windows and PowerShell all write one. A file holding nothing but a BOM is still empty, and is still refused as such.

Found by bug-bashing the off-path cases. A missing dataset reported "has no versions", which is not what happened -- a dataset cannot exist with no versions, and the service answers an unknown name with an empty list rather than a 404. Listing its versions was worse: it said "No datasets found", as though the project had none at all. Both now say the name is not a dataset and point at the listing, matching how the evaluations extension already answers the same question. A name with a space took a file upload and then a 400 carrying four levels of nested JSON, with the sentence that matters at the bottom. The rule is simple and documented, so it is checked before the round trip.

A mistyped path surfaced as 'GetFileAttributesEx <path>: The system cannot find the file specified.' The syscall name is noise to the person who mistyped it.

--from-file <file> resolved to the file's DIRECTORY, and the upload helper then took whichever .jsonl sorted first. Pointing at one dataset in a folder holding several registered a different one under that name. A test pinned the old behaviour: 'a file resolves to the directory the upload scans'.

The unit tests cover the client layer and the helpers; nothing exercised the command surface a user touches - flag parsing, exit codes, the rendered table, and whether -o json emits something a script can consume. 18 tests behind the live tag, gated on AZURE_AI_DATASET_E2E_LIVE, type-checked by ci-test.ps1 like the eval extension's suites.

azure-pipelines · 2026-08-10T04:26:08Z

Azure Pipelines: 7 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run.

Copilot

Pull request overview

Adds a standalone Foundry dataset extension with CRUD/versioning commands, API clients, tests, documentation, and release automation.

Changes:

Adds dataset creation, updates, listing, inspection, deletion, and version listing.
Adds endpoint resolution, validation, structured errors, and JSON/table output.
Adds unit/live tests plus build, lint, and release infrastructure.

Reviewed changes

Copilot reviewed 49 out of 50 changed files in this pull request and generated 19 comments.

Show a summary per file

File	Description
`eng/pipelines/release-ext-azure-ai-dataset.yml`	Adds extension release pipeline.
`cli/azd/extensions/azure.ai.dataset/version.txt`	Sets initial beta version.
`cli/azd/extensions/azure.ai.dataset/tests/cli/harness_test.go`	Adds live CLI harness.
`cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go`	Tests dataset CLI workflows.
`cli/azd/extensions/azure.ai.dataset/README.md`	Documents installation and commands.
`cli/azd/extensions/azure.ai.dataset/main.go`	Adds extension entry point.
`cli/azd/extensions/azure.ai.dataset/internal/version/version.go`	Defines build metadata.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/version_test.go`	Tests version handling.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/uri_test.go`	Tests URI resolution.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/upload_version_test.go`	Tests conflict-aware uploads.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go`	Implements dataset and storage operations.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations_wire_test.go`	Tests API request contracts.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go`	Defines API models and file handling.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go`	Implements listing and version ordering.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_wire_test.go`	Tests download HTTP behavior.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/download_test.go`	Tests blob selection logic.
`cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/bom_test.go`	Tests BOM and file handling.
`cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go`	Centralizes user-facing messages.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/validator.go`	Validates Foundry endpoints.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/types.go`	Defines endpoint-resolution types.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/store.go`	Reads shared project context.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver.go`	Implements endpoint precedence.
`cli/azd/extensions/azure.ai.dataset/internal/foundry/projectctx/resolver_test.go`	Tests endpoint resolution.
`cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go`	Adds structured error helpers.
`cli/azd/extensions/azure.ai.dataset/internal/exterrors/codes.go`	Defines structured error codes.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/table_test.go`	Tests table rendering.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/surface_test.go`	Tests command surface and flags.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/root.go`	Builds the command tree.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go`	Implements JSON and text output.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/output_test.go`	Tests output formatting.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/names.go`	Validates dataset names.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/names_test.go`	Tests naming constraints.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/helpers_test.go`	Tests command helpers.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/debug.go`	Configures debug logging.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go`	Implements dataset commands.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/context.go`	Creates command service context.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/artifacts.go`	Defines persistence helpers.
`cli/azd/extensions/azure.ai.dataset/internal/cmd/apiversions.go`	Defines service API versions.
`cli/azd/extensions/azure.ai.dataset/go.mod`	Defines the Go module.
`cli/azd/extensions/azure.ai.dataset/extension.yaml`	Declares extension metadata.
`cli/azd/extensions/azure.ai.dataset/cspell.yaml`	Configures spell checking.
`cli/azd/extensions/azure.ai.dataset/ci-test.ps1`	Adds CI test execution.
`cli/azd/extensions/azure.ai.dataset/ci-build.ps1`	Adds CI binary builds.
`cli/azd/extensions/azure.ai.dataset/CHANGELOG.md`	Records the initial release.
`cli/azd/extensions/azure.ai.dataset/build.sh`	Adds Unix cross-platform builds.
`cli/azd/extensions/azure.ai.dataset/build.ps1`	Adds Windows cross-platform builds.
`cli/azd/extensions/azure.ai.dataset/.golangci.yaml`	Configures Go linting.
`cli/azd/extensions/azure.ai.dataset/.gitignore`	Ignores generated artifacts.
`.github/workflows/lint-ext-azure-ai-dataset.yml`	Adds GitHub lint workflow.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+    "-X 'azureaieval/internal/version.Version=$Version' " +
+    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
+    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "


+    )
+fi
+
+VERSION_PATH="azureaieval/internal/version"


+    )
+}
+
+$VERSION_PATH = "azureaieval/internal/version"


+	q.Set("comp", "list")
+	u.RawQuery = q.Encode()
+
+	log.Printf("[dataset_api] listing blobs: %s", u.Redacted())


+func DatasetVersionNotFoundWithHint(dataset, version string) error {
+	return fmt.Errorf(
+		"no dataset %q at version %q in this project; "+
+			"`azd ai dataset list` shows the ones there are", dataset, version)
+}


+	var respErr *azcore.ResponseError
+	if !errors.As(err, &respErr) {


+	var respErr *azcore.ResponseError
+	if errors.As(err, &respErr) {


+	cmd.Flags().StringVar(&version, "version", "",
+		"Current version to increment from. Omit to increment from the latest registered version.")


+- The CRUD groups moved here from `azure.ai.evaluations`. Generation stayed
+  there: `dataset generate` writes the `datasets:` entry in `evals/eval.yaml`,
+  and that file belongs to the evaluation extension.


Two lines of ci-test.ps1 assumed PowerShell 6 or later. $IsWindows does not exist on Windows PowerShell 5.1, so the .exe suffix was never appended; and Join-Path there takes a single child path, so the three-argument call failed outright. The script then found no gotestsum, printed the fallback notice, and ran bare `go test` with no JUnit report -- while still exiting 0, which is why it went unnoticed.

Copilot

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 1 comment.

Suppressed comments (16)

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

The release build stamps azureaieval, but this module is azureaidataset. These -X flags therefore do not update the variables used by the dataset binary, leaving release requests identified with the default dev version. Point all three symbols at this module's actual import path.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

This developer build script still targets the evaluation module's version package. As a result, binaries built through this path retain the dataset package's default build metadata. Use the azureaidataset module path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This PowerShell build path also stamps azureaieval instead of this module, so locally packaged dataset binaries do not receive their version, commit, or build date. Use the dataset module path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:459

url.URL.Redacted() only redacts userinfo passwords; it leaves query parameters intact. Here the query contains the SAS sig, so every container listing writes a reusable storage credential to the debug log. Log a copy with its query removed.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19
The API response is explicitly paged, but both list methods return only the first Value and never follow NextLink. Projects beyond the service page limit will silently omit datasets or versions; this can also make show select an old version and update calculate the wrong next version. Follow links until empty and return the combined values.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:79
Any listing failure is converted into exists == false. An update during a 401, 429, or network failure therefore reports that the dataset does not exist, while create proceeds toward an upload. Return the list error; only run the consistency probe after a successful empty list, and only treat 404 probe results as absence.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105

The PR describes CRUD as configuration-free, but every create/update overwrites a project-wide EVAL_DATASET_VERSION. The key contains no dataset name, so publishing any second dataset clobbers the first dataset's value, and this extension never reads the cache. Remove this persistence (and its dead helpers), or define a per-dataset contract with an actual consumer.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/README.md:29

The linked evaluation extension defines the configuration as azure.eval.yaml (while only preserving an existing legacy eval.yaml), so documenting evals/eval.yaml as the owned file is stale and sends users to the wrong path. Update this reference to the actual configuration location.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that
extension's file. Splitting the two would leave a generated dataset registered

cli/azd/extensions/azure.ai.dataset/README.md:41

This documented cascade omits two implemented levels: the shared global project context and the AZURE_AI_PROJECT_ENDPOINT fallback at both environment scopes. A user can therefore see behavior that contradicts the stated order. Document all four sources in the same order as projectctx.Resolve.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging now creates azd-ai-dataset-*.log, but the ignore rule still names the old evaluation log. Running any command with debug enabled will leave the generated credential-bearing diagnostic file visible to Git and at risk of accidental commit.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/README.md:6

This getting-started command cannot work in this PR: the TODO below states that the registry entry is deliberately absent, so azd extension install azure.ai.dataset cannot resolve. Document the local azd x pack/publish/install flow here until the registry entry lands, rather than opening with a guaranteed failure.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:3

The release history is labeled beta.1 while both extension.yaml and version.txt declare beta.5. This makes the changelog inaccurate for the artifact being prepared; keep the heading synchronized with the release version.

## 1.0.0-beta.1 (Unreleased)

cli/azd/extensions/azure.ai.dataset/internal/cmd/output.go:34

Every non-json value is silently treated as human output, and none of these commands registers allowed values with the extension SDK. Consequently -o yaml succeeds but emits a table instead of rejecting an unsupported format. Register output options (json, table, with table as the default) on each runnable command, as the sibling Foundry extensions do.

// isJSON reports whether the command should emit machine-readable output.
func isJSON(cmd *cobra.Command) bool {
	return outputFormat(cmd) == outputJSON

cli/azd/extensions/azure.ai.dataset/README.md:22

This documented invocation always fails because the implementation requires --version before making a request. Include the required flag so the command table is directly runnable.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:31

The PR states that every user-facing string is centralized in internal/messages/messages.go, but command descriptions are introduced here, with additional flag help and table labels elsewhere in this package. Either move these strings into the messages package or revise that stated contract; the implementation does not provide the promised single review surface.

	return newDatasetWriteCommand("create", "Register a dataset, publishing its first version.")

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:47

Cleanup silently returns when listing or JSON decoding fails, and delete exit codes are also ignored below. The live suite can pass while leaving remote dataset versions behind, making later runs increasingly stateful. Report cleanup failures and verify deletes while continuing best-effort cleanup.

+	list, err := c.ListDatasetVersions(ctx, name, apiVersion)
+	if err != nil || list == nil || len(list.Value) == 0 {
+		return ""


The heading still said 1.0.0-beta.1 while version.txt and extension.yaml say 1.0.0-beta.5. Both "Release" commits bumped those two files and left the changelog alone, so it has not moved since the extension was added. The release pipeline reads the entry for the version it builds. Running eng/scripts/New-ReleaseNotesFile.ps1 the way the pipeline does produced the literal string "[Missing change log entry]" and exit code 0, so a release would have published that quietly rather than failing. The fixes made since that first entry had never been recorded, so they are listed too.

Copilot

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated 1 comment.

Suppressed comments (15)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105

[azd-code-reviewer] This writes an eval-specific, dataset-name-free value into the active azd environment after every create/update, even though the PR states CRUD requires no configuration. Creating any second dataset overwrites the same value, and this extension never reads it. I think we need to remove this persistence so standalone dataset CRUD does not mutate evaluation state.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19

[azd-code-reviewer] Both list methods deserialize nextLink but return after the first page. This makes dataset list and versions list incomplete, and show/update can select an old version when the newest page was not fetched. Follow all same-origin continuation links before returning the combined Value slice.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:282
[azd-code-reviewer] On transport failure, http.Client.Do returns a url.Error whose text includes the complete request URL. Because this URL carries the upload SAS query, wrapping the error can expose sig in terminal output and telemetry. Sanitize the URL query before returning the error.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:427
[azd-code-reviewer] This download error also embeds the full SAS URL, including sig, when Do fails before receiving a response. Redact the query from the returned url.Error before it reaches user output or telemetry.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:469
[azd-code-reviewer] A container-list transport error includes the full request URL, so wrapping it exposes the SAS sig. Sanitize the query in the returned url.Error before surfacing it.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:507
[azd-code-reviewer] The same SAS disclosure occurs here on a blob-download transport failure: url.Error retains the signed query and is returned to the user. Strip the query before wrapping the error.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45
[azd-code-reviewer] These linker targets still use the evaluation module path, while this module is azureaidataset. Go silently ignores -X targets that do not exist, so release binaries built by the new pipeline keep version.Version == "dev" and emit the wrong User-Agent version. Stamp the dataset module symbols instead.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

[azd-code-reviewer] The local PowerShell build stamps symbols in azureaieval, which is not part of this module. The build succeeds but leaves the dataset binary reporting dev; use the azureaidataset module path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

[azd-code-reviewer] The shell build also targets the nonexistent azureaieval symbols, so packaged binaries are not version-stamped even though the build succeeds. Use this module's import path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:80

[azd-code-reviewer] A failed list request is currently converted into exists == false. In particular, update reports “dataset does not exist” during an auth, network, or service failure instead of surfacing the real error. Only an empty successful list should enter the eventual-consistency probes.

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:170

[azd-code-reviewer] Treating every HTTP 409 as “version already exists” masks unrelated conflicts and retries them against four different version numbers. The surrounding contract names TemporaryDataReferencesForExistingAsset; check that response error code as well so other conflicts are returned immediately.
cli/azd/extensions/azure.ai.dataset/README.md:6
[azd-code-reviewer] This quick start cannot work in the proposed state: the same README notes that no registry entry exists, so azd extension install azure.ai.dataset cannot resolve. Replace it with complete local pack/publish instructions until the registry entry lands.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

[azd-code-reviewer] The documented delete invocation omits the required --version flag, so copying it immediately fails with the command's required-value error.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/README.md:41

[azd-code-reviewer] This resolution order is incomplete. The implementation also checks AZURE_AI_PROJECT_ENDPOINT as a fallback at both environment levels and checks the shared global project context before host variables. Documenting only three levels makes it hard to diagnose which endpoint a command selected.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

[azd-code-reviewer] Debug logging creates azd-ai-dataset-*.log, but this copied pattern ignores only evaluation logs. Running the new extension with --debug therefore leaves its credential-sensitive diagnostic file unignored.

azd-ai-eval-*.log

+	q.Set("comp", "list")
+	u.RawQuery = q.Encode()
+
+	log.Printf("[dataset_api] listing blobs: %s", u.Redacted())


`go fix` rewrote three `strings.Split` range loops to `strings.SplitSeq`, which is why the go-fix check failed: it reports a diff rather than applying one. Applying it makes the check a no-op, and the suite still passes.

Copilot

Pull request overview

Copilot reviewed 53 out of 54 changed files in this pull request and generated no new comments.

Suppressed comments (19)

cli/azd/extensions/azure.ai.dataset/README.md:28

This points users to a command that the evaluation extension does not expose. Its surface is azd ai eval generate, with --dataset selecting dataset generation; using the current text results in an unknown-command error.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/build.sh:36

The Unix extension build stamps azureaieval, which is not this module. Point the linker at azureaidataset/internal/version so locally packed binaries carry the requested version and build identity.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

The debug logger writes azd-ai-dataset-*.log, but this rule ignores the old evaluation filename. Dataset logs—including request diagnostics—will remain unignored and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

This names the evaluation extension's opt-in variable, but tests/cli/TestMain checks AZURE_AI_DATASET_E2E_LIVE. Following this instruction silently skips the live suite instead of running it.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/README.md:41

The documented endpoint cascade omits the AZURE_AI_PROJECT_ENDPOINT fallback and the shared global project context that projectctx.Resolve checks before host variables. Users may unnecessarily set a lower-precedence value because this order does not match runtime behavior.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker symbols still target the evaluation module. The release template invokes this script for every artifact, so azureaidataset/internal/version.Version, Commit, and BuildDate are never stamped and the dataset User-Agent remains at its dev default.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.ps1:44

The PowerShell extension build stamps azureaieval, which is not this module. Point the linker at azureaidataset/internal/version so locally packed binaries carry the requested version and build identity.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:459

url.URL.Redacted() only hides user-info passwords; it does not remove query parameters. This logs the full container SAS, including sig, into the debug file. Log only the scheme/host/path and keep the authenticated URL solely on the request.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19
Both list methods discard NextLink, so projects with a paged response silently lose datasets/versions. This also makes show and update-version discovery select an older version. Follow all pages and validate each continuation URL is same-origin before requesting it.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:79
A list failure is currently converted into exists == false. update then reports that the dataset does not exist for authentication, transport, and service failures, while create proceeds toward an upload. Propagate the list error; only a successful empty response means absent.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:88

The consistency probes ignore every error as if the version were absent. If either point read fails with auth, transport, or a 5xx response, update returns the misleading “does not exist” error. Continue only on 404 and propagate other failures.

					if _, err := ec.datasetClient.GetDataset(
						ctx, name, v, ProjectEndpointAPIVersion,
					); err == nil {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105

The PR states that CRUD needs no configuration, but every create/update mutates the active environment by writing one global EVAL_DATASET_VERSION. That key cannot represent multiple datasets and has no reader in this extension, so the command introduces an unrelated persistent side effect. Remove this write and its dead cache helpers.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:157

This collapses every listing error into “no registered version” and then attempts to publish 1.0. A transient list failure can therefore create an old version or turn the real service error into a conflict. Return the error separately; reserve the empty value for a successful empty listing.
cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go:94
Use errors.AsType[*azcore.ResponseError](err) here. This repository's Go 1.26 modernization rules prohibit the target-variable errors.As pattern, and CI runs go fix -diff.

	var respErr *azcore.ResponseError
	if errors.As(err, &respErr) {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:177

Use the Go 1.26 errors.AsType form required by this repository; the current target-variable pattern is subject to the enforced go fix modernization check.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:168
This module targets Go 1.26, and the repository requires errors.AsType for typed error matching. The go fix -diff workflow will reject this older target-variable pattern.
cli/azd/extensions/azure.ai.dataset/README.md:6
This quickstart cannot currently succeed: the same README notes that no registry entry exists, so azd extension install azure.ai.dataset cannot resolve. Document the local pack/publish flow until the registry entry lands, or add the registry entry before presenting this command.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete command omits the required --version flag, so copying it always fails with --version is required. Include the mandatory flag in the command table.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:304

The code already documents that this version listing can return empty for a few seconds after creation, but show treats that stale response as proof the dataset does not exist. An immediate create → show can therefore fail falsely; use a bounded consistency fallback before returning not-found.

				if len(list.Value) == 0 {
					return messages.DatasetNotFound(name)
				}

cspell: - Correct "neighbouring" to "neighboring" in the changelog. - Add the identifiers cspell cannot infer to the extension dictionary: azureaidataset, projectctx, projectctx's, exterrors, CLI's. These mirror the entries the azure.ai.evaluations extension already carries. golangci-lint: - Delete datasetContext.getEnvValue and the surface_test find helper. Both had no callers, so they were dead rather than suppressible. - Rename the storageServer test field credentialPath to uriPath. It holds a URL path, never a secret; the old name is what tripped gosec G101, so the rename removes the finding instead of hiding it behind a nolint. - Annotate the three filepath.WalkDir ReadFile calls in surface_test with //nolint:gosec and the reason: these walk this package's own source, not caller input. No behaviour change. go build, go vet, gofmt and go test ./... all clean.

Seven review comments flagged this and were all marked outdated as line numbers moved, so none of them had actually been applied. `url.URL.Redacted()` masks a userinfo password only; it leaves the query string intact. A storage SAS carries its credential in the query as `sig`, so both `log.Printf(... u.Redacted())` sites wrote a live, still-valid storage credential into `azd-ai-dataset-*.log` whenever --debug was on. Separately, `http.Client.Do` returns a *url.Error whose message embeds the full request URL. For the three SAS-backed calls (blob upload, container listing, blob download) a DNS/TLS/timeout/cancellation failure therefore printed the SAS to the user. - Add logSafeURL, which drops the query and fragment before formatting. - Add redactURLError, which rebuilds a *url.Error with a redacted URL and without mutating the original. Uses errors.AsType per cli/azd/AGENTS.md. - Apply to both log sites and all three SAS-backed Do() calls. Tests assert the premise as well as the fix: one case pins that Redacted() alone does leak the signature, so the unsafe call cannot quietly come back. The request itself still uses the unmodified URL. gofmt, go vet, go build and go test ./... are clean.

Copilot

Pull request overview

Copilot reviewed 54 out of 55 changed files in this pull request and generated 3 comments.

Suppressed comments (20)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:427

A failed direct download returns a raw *url.Error, which includes the full SAS-bearing request URL and can expose the token in user-facing output or the fallback debug log. Redact the query before wrapping this error. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:469
A container-list transport error includes the full SAS URL in *url.Error.Error(), so returning it through ListingContainerBlobs can expose the token. Sanitize the query before wrapping the error. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:507
A failed blob download returns a raw *url.Error containing the full SAS-bearing URL. That error is rendered to users, so redact its query before wrapping it. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19
NextLink is captured but never followed, so list, versions list, latest-version resolution, and existence checks only see the first page. The methods promise all datasets/versions and the JSON renderer deliberately drops the paging link, leaving callers no way to fetch omitted pages. Follow pages inside the client before returning. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105
Every dataset publish overwrites the same project-wide EVAL_DATASET_VERSION, even though the command can manage many independently named datasets. Publishing dataset B can therefore leave consumers treating B's version as A's, and this extension never reads the key back. Remove this mutation or persist a dataset-keyed value with a documented consumer contract. — azd-code-reviewer

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker targets still use the evaluation module path. Because this module is azureaidataset, the release build leaves Version, Commit, and BuildDate at their defaults, including the dev version used in the User-Agent. Point all three targets at this module. — azd-code-reviewer

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

This copied module path prevents local cross-platform builds from stamping the dataset extension's version metadata; the linker target must match the azureaidataset module. — azd-code-reviewer

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This copied module path prevents the PowerShell build from stamping this extension's version metadata; target the azureaidataset module instead. — azd-code-reviewer

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:96

This turns every list failure—401/403, timeout, or 5xx—into exists == false, then also ignores non-404 point-read failures. update consequently reports “does not exist,” while create may attempt an upload without establishing that the name is free. Return list errors and only ignore a point-read error when it is actually not-found. — azd-code-reviewer

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:304

This treats an empty version list as authoritative even though the new upload path explicitly documents that this listing is eventually consistent. A show immediately after a successful create can therefore report that the dataset does not exist. Resolve the just-published version through a consistent point read or retry the empty listing before returning not-found. — azd-code-reviewer

				if len(list.Value) == 0 {
					return messages.DatasetNotFound(name)
				}

cli/azd/extensions/azure.ai.dataset/.gitignore:5

The generated debug file is named azd-ai-dataset-<date>.log (internal/cmd/debug.go:31), so this copied evaluation pattern does not ignore it and makes credential-bearing diagnostics easy to commit accidentally. — azd-code-reviewer

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/.golangci.yaml:12

This disables the repository's established 125-character Go limit for the new module; the root configuration sets it at cli/azd/.golangci.yaml:27. Keeping 220 here means this extension's dedicated lint workflow accepts code the rest of azd rejects. — azd-code-reviewer

      line-length: 220

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

The live harness actually requires AZURE_AI_DATASET_E2E_LIVE; this copied evaluation variable sends anyone following the CI instructions to a run that silently skips all tagged tests. — azd-code-reviewer

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

The live test and stated behavior require delete to be idempotent, but a service that represents an already-absent version as 404 is converted into a failure here. Treat not-found as successful deletion and only return other errors. — azd-code-reviewer

				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:238

This now makes an unknown name return an empty successful list, while the PR description still says this extension errors and presents the behavior as an open question. Update the description to record the decision implemented and tested here. — azd-code-reviewer

			// An unknown name lists nothing and succeeds; it is not an error.
			// `-o json` callers range over the array, and a delete is checked
			// for idempotence by listing what is left. The empty sentence names
			// the dataset, though: the project may hold plenty of others, so
			// "No datasets found." would be answering a different question.
			return renderDatasets(cmd, list, messages.NoDatasetVersions(name))

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

Cleanup silently returns on list/JSON failures and ignores every delete result, so the live suite can pass while leaving all created dataset versions behind. Make cleanup failures visible and require each delete to succeed. — azd-code-reviewer
cli/azd/extensions/azure.ai.dataset/README.md:6
This first-run command cannot work in this PR because the registry entry is deliberately omitted, as the TODO at lines 54–57 confirms. Replace it with complete local-registry setup instructions or defer the install example until the extension is resolvable. — azd-code-reviewer

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

delete rejects calls without --version, but the command table presents this incomplete invocation. Include the required flag so the documented command succeeds. — azd-code-reviewer

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/README.md:41

This claims a three-level endpoint cascade, but the resolver also checks AZURE_AI_PROJECT_ENDPOINT at the azd/host levels and the shared global project context between them. Document all levels so users can understand why an endpoint other than these three wins. — azd-code-reviewer

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/README.md:28

The companion evaluation extension exposes azd ai eval generate, not azd ai eval dataset generate, and its current configuration is azure.eval.yaml. As written, this sends users to a nonexistent command and names the wrong file. — azd-code-reviewer

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

+	resp, err := httpClient.Do(req)
+	if err != nil {
+		return messages.UploadingBlobFailed(err)


+	q.Set("comp", "list")
+	u.RawQuery = q.Encode()
+
+	log.Printf("[dataset_api] listing blobs: %s", u.Redacted())


+	best := ""
+	bestOrder := -2.0
+	for _, d := range datasets {
+		if o := VersionOrder(d.Version); o > bestOrder {
+			bestOrder, best = o, d.Version
+		}
+	}
+	if best == "" && len(datasets) > 0 {
+		return datasets[len(datasets)-1].Version
+	}


The cspell job runs `--config ./.vscode/cspell.yaml`, so words have to be known to the repo-level config; the extension's own cspell.yaml serves the other job and the editor. - "unauthorised" -> "unauthorized" in a ServiceRefused comment. US spelling is what the rest of the repo uses. - Rename the redactURLError local from `uerr` to `urlError`. `uerr` is not a word cspell can resolve, and naming it properly removes the finding rather than adding a dictionary entry for an abbreviation. - Register CODEOWNERS for the extension README under `overrides`, matching how this config already scopes per-file words. Verified with the exact CI invocation: 577 files checked, 0 issues.

Copilot

Pull request overview

Copilot reviewed 55 out of 56 changed files in this pull request and generated 1 comment.

Suppressed comments (16)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:24

[azd-code-reviewer] Both list methods return only the first page even though DatasetList exposes NextLink. Projects with enough datasets or versions will omit entries; show can select the wrong latest version and update can derive a conflicting version. Follow same-origin pagination links until exhausted, as the repository's connection clients do.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:157
[azd-code-reviewer] latestRegisteredVersion collapses every list error into an empty version. On an auth or service failure, UploadNextVersion then attempts 1.0 and can return a misleading conflict instead of the original failure. Return (string, error) and propagate errors from both call sites; an empty successful list already represents an unknown or not-yet-consistent dataset.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:105
[azd-code-reviewer] CRUD is described as configuration-free, but every create/update writes one global EVAL_DATASET_VERSION into the active azd environment. That value has no dataset name, and --project-endpoint may target a project unrelated to the active environment, so publishing any dataset can overwrite evaluation state with an ambiguous or foreign version. Remove this persistence from the standalone dataset extension.

			if err := ec.setEnvValue(ctx, envKeyDatasetVersion, ds.Version); err != nil {

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:194

[azd-code-reviewer] This module targets Go 1.26, and the repository requires errors.AsType rather than the mutable-target errors.As pattern. Use the typed helper here.
cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:203
[azd-code-reviewer] Use the repository's Go 1.26 errors.AsType pattern here instead of declaring a mutable target for errors.As.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45
[azd-code-reviewer] The release build stamps symbols in the evaluation module, which is not part of this binary. The linker therefore leaves version.Version as dev, so every published request still uses azd-ext-azure-ai-dataset/dev despite the identity fix. Stamp the azureaidataset module instead.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

[azd-code-reviewer] Any listing failure is currently treated as “dataset absent.” A 401, 403, timeout, or 5xx therefore makes update report that the dataset does not exist and lets create continue toward an upload. Propagate list errors, and ignore only IsNotFound from the consistency probes.

			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/build.ps1:44

[azd-code-reviewer] This local PowerShell build also targets the evaluation module, so its version, commit, and date linker values are silently ignored. Use this extension's module path.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.sh:36

[azd-code-reviewer] This local shell build targets the evaluation module, leaving the dataset binary's build metadata at its development defaults. Use this extension's module path.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:41

[azd-code-reviewer] The documented cascade omits two implemented sources: the shared project context and the AZURE_AI_PROJECT_ENDPOINT fallback. This makes the stated precedence inaccurate and hides why a command may use an endpoint other than the listed values.

Every command resolves the Foundry project endpoint in this order:

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/README.md:29

[azd-code-reviewer] This points readers to a command that the companion evaluation extension does not expose (#9500 defines azd ai eval generate) and names the obsolete config path. A user following this section gets an unknown command before reaching dataset creation.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that
extension's file. Splitting the two would leave a generated dataset registered

cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10

[azd-code-reviewer] The live harness checks AZURE_AI_DATASET_E2E_LIVE, not this evaluation variable. Following this CI documentation causes the suite to print its skip message and exit successfully instead of running.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

cli/azd/extensions/azure.ai.dataset/.gitignore:5

[azd-code-reviewer] Debug logging now writes azd-ai-dataset-*.log, so this inherited evaluation pattern does not ignore the generated files. Running with --debug leaves untracked logs in the extension directory.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/internal/exterrors/errors.go:95

[azd-code-reviewer] Use errors.AsType[*azcore.ResponseError] here; the repository's Go 1.26 modernization rules prohibit the mutable-target errors.As form.
cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51
[azd-code-reviewer] Cleanup silently returns on list/JSON failures and ignores every delete result, so a green live run can leave dataset versions in the shared Foundry project. Make cleanup failures fail the test and show the command output instead of hiding leaked remote state.
cli/azd/extensions/azure.ai.dataset/CHANGELOG.md:11
[azd-code-reviewer] The companion extension exposes azd ai eval generate, not dataset generate. The release notes currently document a command users cannot run.

- The CRUD groups moved here from `azure.ai.evaluations`. Generation stayed
  there: `dataset generate` writes the `datasets:` entry in `evals/eval.yaml`,
  and that file belongs to the evaluation extension.

+	httpClient := blobHTTPClient()
+	resp, err := httpClient.Do(req.Raw())
+	if err != nil {
+		return nil, messages.DownloadingDatasetBlob(err)


Three review findings, all still live in the code despite being marked outdated when line numbers moved. latestRegisteredVersion swallowed every listing error and returned "", which is indistinguishable from a brand-new dataset. A 403, throttle, timeout or cancellation therefore restarted an existing dataset at 1.0 and issued a pending upload against it. It now returns (string, error): a 404 and an empty listing still report "" with no error, everything else is propagated. The retry loop keeps best-effort semantics, because the version it just had refused is already a correct next step. LatestVersion documented a fallback to the service's last entry when nothing is orderable, but bestOrder started at -2.0 and VersionOrder returns -1 for an unorderable version, so the *first* one always won and the fallback was unreachable. Starting at -1.0 makes the documented behaviour real. Also finishes the errors.As -> errors.AsType migration required by cli/azd/AGENTS.md; no errors.As calls remain in the extension. Tests cover all three, and were checked by reverting each fix: - a refused listing must surface and must not reach startPendingUpload - an unknown dataset (404) still starts at 1.0 - an all-unorderable list returns the last entry, not the first

The paged response modelled nextLink but neither list method followed it, so a project with more than one page listed incompletely and latestRegisteredVersion could pick a version from a stale first page. followPages walks the links and returns one merged list. Two guards come with it: - Same-origin check. nextLink is service-supplied and the pipeline attaches the caller's token, so a link off the endpoint's origin is refused rather than followed; otherwise a spoofed or redirected response could hand that token to another host. - Repeat and count cap. A service that keeps returning the same link stops the walk instead of spinning; that is the service misbehaving, not a reason to fail the user's command. Also corrects the envKeyDatasetVersion comment. It claimed the key caches a version "so a later read does not have to list every version" — no code in either extension reads it, so the comment described an optimization that does not exist. Tests: pages merge in order, an off-origin link fails the call and the other host is never contacted, and a repeated link terminates.

Copilot

Pull request overview

Copilot reviewed 57 out of 58 changed files in this pull request and generated 1 comment.

Suppressed comments (17)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/list.go:19

Both list methods stop after the first response even though this model exposes NextLink. In a paged project, list is incomplete and LatestVersion can select an old version, causing update to target the wrong version. Follow all continuation links and validate that each link has the same origin as the configured endpoint.
cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93
The list error is converted into exists == false. For update, a 401, 403, or 5xx is then swallowed along with both probe errors, and the user receives “does not exist” instead of the authentication/service failure. Propagate non-404 failures from both the listing and point probes.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0
			if !exists {

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

An unknown dataset can be represented by a 404—the same client already handles that case in latestRegisteredVersion. Returning every error here contradicts the documented/tested contract that versions list <unknown> succeeds with an empty result.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:298

ListDatasetVersions is documented elsewhere in this client as eventually consistent, but show reads it only once. Immediately after a successful create or update, an empty stale page therefore becomes a false “dataset not found” error. Add a bounded retry/backoff for an empty latest-version listing before concluding the dataset is absent.

			if version == "" {
				list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
				if err != nil {
					return messages.ResolvingLatestDatasetVersion(name, err)
				}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:366

The command converts a 404 into failure even though the CLI contract and TestCLIDeleteIsIdempotent require deleting an absent version to succeed. Ignore not-found responses and emit the normal idempotent success result; only other failures should be returned.

			); err != nil {
				if dataset_api.IsNotFound(err) {
					return messages.DatasetVersionNotFound(name, version)
				}
				return messages.DeletingDatasetVersion(name, version, err)

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/models.go:51

These wire-format aliases are also serialized directly by every -o json command. A response using snake_case therefore exposes data_uri, while the same dataset from another endpoint exposes dataUri (and a mixed response can expose both), breaking the stable script-facing shape asserted by the CLI tests. Decode aliases internally, then map to a dedicated canonical output model before rendering JSON.
cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45
The release pipeline invokes this script, but these linker paths still target the evaluation module. The dataset module’s version variables are never stamped, so released requests keep the azd-ext-azure-ai-dataset/dev User-Agent instead of the published version.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

This copied linker path targets azureaieval, so locally packaged binaries do not stamp this module’s Version, Commit, or BuildDate. Point it at the module declared by this extension.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

This copied linker path targets azureaieval, so locally packaged binaries do not stamp this module’s Version, Commit, or BuildDate. Point it at the module declared by this extension.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/README.md:6

This quickstart cannot currently work: the same README states that no registry entry exists, so azd extension install azure.ai.dataset cannot resolve. Document the local pack/publish installation path here, or defer this install example until the registry entry lands.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete invocation omits the required --version flag, so copying it always fails with “--version is required.”

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/README.md:28

The companion evaluation surface exposes azd ai eval generate, not azd ai eval dataset generate, and its default configuration is azure.eval.yaml (with compatibility for an existing eval.yaml), not evals/eval.yaml. These instructions currently lead users to a nonexistent command and file.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This list omits two implemented resolution sources and the AZURE_AI_PROJECT_ENDPOINT fallback, so users may not understand why a different endpoint won. Document the full cascade implemented by projectctx.Resolve.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging creates azd-ai-dataset-<date>.log, but this copied pattern ignores only evaluation logs. Dataset debug logs will therefore appear as untracked files and can be committed accidentally.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/tests/cli/dataset_test.go:51

Cleanup silently returns when listing or JSON decoding fails, and delete failures are also ignored. A live test can therefore pass while leaving every created dataset version in the shared Foundry project. Fail/report these operations so leaked test assets are visible.
cli/azd/extensions/azure.ai.dataset/internal/messages/messages.go:195
This error is specifically about a missing version, but it sends the user to dataset list, which does not enumerate that dataset’s versions. Point to dataset versions list <name> so the suggested command can actually find a valid version.
cli/azd/extensions/azure.ai.dataset/ci-test.ps1:10
The live suite is gated by AZURE_AI_DATASET_E2E_LIVE, not the copied evaluation variable named here. Anyone following this CI documentation will set the wrong variable and the suite will exit without running.

# AZURE_AI_EVAL_E2E_LIVE and a project endpoint. They are still type-checked

Copilot

Pull request overview

Copilot reviewed 57 out of 58 changed files in this pull request and generated 1 comment.

Suppressed comments (12)

cli/azd/extensions/azure.ai.dataset/ci-build.ps1:45

These linker overrides still target the evaluation module, so the release pipeline silently leaves this binary's Version, Commit, and BuildDate at dev/none/unknown. Stamp azureaidataset/internal/version instead so the dataset User-Agent carries the released identity.

    "-X 'azureaieval/internal/version.Version=$Version' " +
    "-X 'azureaieval/internal/version.Commit=$SourceVersion' " +
    "-X 'azureaieval/internal/version.BuildDate=$(Get-Date -Format o)' "

cli/azd/extensions/azure.ai.dataset/build.sh:36

The local shell build stamps azureaieval, which is not this module, so dataset binaries keep the fallback version metadata. Point the linker path at this module.

VERSION_PATH="azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/build.ps1:44

The local PowerShell build stamps azureaieval, which is not this module, so dataset binaries keep the fallback version metadata. Point the linker path at this module.

$VERSION_PATH = "azureaieval/internal/version"

cli/azd/extensions/azure.ai.dataset/.gitignore:5

Debug logging creates azd-ai-dataset-<date>.log, but this copied pattern ignores eval logs instead. Running the new extension with --debug therefore leaves its credential-sensitive diagnostic log visible as an untracked file.

azd-ai-eval-*.log

cli/azd/extensions/azure.ai.dataset/README.md:28

Both references are stale relative to the companion evaluation extension: its command surface is azd ai eval generate, and its canonical configuration is azure.eval.yaml (while preserving an existing eval.yaml), not evals/eval.yaml. As written, users are sent to a nonexistent command and file.

Generation is `azd ai eval dataset generate`, in `azure.ai.evaluations`, and
stays there: it writes the `datasets:` entry in `evals/eval.yaml`, which is that

cli/azd/extensions/azure.ai.dataset/README.md:41

This omits two resolution levels implemented by projectctx.Resolve: the shared global project context and AZURE_AI_PROJECT_ENDPOINT fallbacks in both azd and host environments. Document the full precedence order; otherwise users can neither predict which endpoint wins nor discover the supported fallback.

1. `--project-endpoint`
2. `FOUNDRY_PROJECT_ENDPOINT` in the active azd environment
3. the host environment variable of the same name

cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go:515

This direct-download path wraps the raw *url.Error, whose URL contains the SAS sig query. On a transport failure the credential can therefore appear both in the user-facing error and in the debug log at the fallback call site. Apply the same redactURLError used by the other blob operations before wrapping it.
cli/azd/extensions/azure.ai.dataset/README.md:6
This quick start cannot currently work: the PR deliberately omits the registry.json entry, and lines 54-57 state that azd extension install azure.ai.dataset cannot resolve yet. Replace this with the local pack/publish workflow for now, or add the install command only when the registry entry lands.

$ azd extension install azure.ai.dataset

cli/azd/extensions/azure.ai.dataset/README.md:22

The documented delete invocation omits --version, but the implementation rejects every delete without that flag. Include the required flag so copying this command does not immediately fail.

| `azd ai dataset delete <name>` | Delete a dataset version |

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:26

This probe does not cover every first version this CLI can publish. Because create also exposes --version, create --version 4.0 publishes 5.0; during listing lag, the subsequent update probes only 1.0 and 1 and incorrectly reports that the dataset does not exist. Either remove --version from create or include NextVersion(version) in the fallback probes.

var firstDatasetVersions = []string{dataset_api.NextVersion(""), "1"}

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:93

A listing failure is currently converted into exists == false, and every direct-probe error is also ignored. A 403, timeout, or token failure therefore becomes the misleading “does not exist” result for update, while create may continue toward an upload. Only 404 should mean absence; propagate other errors.

			existing, err := ec.datasetClient.ListDatasetVersions(
				ctx, name, ProjectEndpointAPIVersion,
			)
			exists := err == nil && existing != nil && len(existing.Value) > 0

cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go:232

The comment promises that an unknown name returns an empty successful list, but a 404 from the versions endpoint is returned as an error here. The API client already treats 404 as the normal “no versions” case in latestRegisteredVersion; normalize it here too so the command's documented behavior is consistent across service responses.

			list, err := ec.datasetClient.ListDatasetVersions(ctx, name, ProjectEndpointAPIVersion)
			if err != nil {
				return messages.ListingDatasetVersions(name, err)
			}

Mohamed Hessien (m7md7sien) added 7 commits August 9, 2026 17:24

Stop the same panic and leak here

93516a2

Carved from the same code, so it had the same two: a nil Environment or GetValue response panicking inside the endpoint cascade, and an azd client abandoned when endpoint or credential setup failed.

Say when the answer was 'log in again' here too

2e8f8fb

Same credential failure and same 401/403 path as the eval extension, which this was carved from.

Tell people to retry before telling them to log in

88ffc5f

Same intermittent credential failure as the eval extension.

Refuse an empty dataset early, bound the blob calls, guard the nil list

dd0b293

Same three as the eval extension, which this was carved from.

microsoft-github-policy-service Bot added the customer-reported identify a customer issue label Aug 10, 2026

Mohamed Hessien (m7md7sien) added 2 commits August 10, 2026 03:50

Mohamed Hessien (m7md7sien) mentioned this pull request Aug 10, 2026

Add azd ai eval extension for Foundry evaluations #9500

Open

Mohamed Hessien (m7md7sien) added 4 commits August 10, 2026 05:52

Format the Go sources with gofmt

9df2b57

Say a --from-file path is missing, not which syscall found out

be18334

A mistyped path surfaced as 'GetFileAttributesEx <path>: The system cannot find the file specified.' The syscall name is noise to the person who mistyped it.

Mohamed Hessien (m7md7sien) marked this pull request as ready for review August 10, 2026 04:25

Mohamed Hessien (m7md7sien) requested review from Daniel Jurek (danieljurek), tg-msft and Victor Vazquez (vhvb1989) as code owners August 10, 2026 04:26

Copilot AI balanced review requested due to automatic review settings August 10, 2026 04:26

Mohamed Hessien (m7md7sien) requested review from JeffreyCA, Rick Winter (RickWinter), Marina He (hemarina) and Richard Park (richardpark-msft) as code owners August 10, 2026 04:26

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 10, 2026 04:26 View session

Copilot AI reviewed Aug 10, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 10, 2026 05:42

Copilot AI review requested due to automatic review settings August 11, 2026 18:53

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 11, 2026 18:54 View session

Copilot AI reviewed Aug 11, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go Outdated

Comment on lines +155 to +157

list, err := c.ListDatasetVersions(ctx, name, apiVersion)

if err != nil || list == nil || len(list.Value) == 0 {

return ""

Mohamed Hessien (m7md7sien) added area/evals Evaluations, benchmarks, quality checks and removed customer-reported identify a customer issue labels Aug 12, 2026

Copilot AI review requested due to automatic review settings August 12, 2026 20:28

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 12, 2026 20:29 View session

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go Outdated

q.Set("comp", "list")

u.RawQuery = q.Encode()

log.Printf("[dataset_api] listing blobs: %s", u.Redacted())

Copilot AI review requested due to automatic review settings August 12, 2026 21:10

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 12, 2026 21:10 View session

Copilot AI reviewed Aug 12, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 13, 2026 01:44

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 01:45 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings August 13, 2026 01:54

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 01:55 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go

httpClient := blobHTTPClient()

resp, err := httpClient.Do(req.Raw())

if err != nil {

return nil, messages.DownloadingDatasetBlob(err)

Copilot AI review requested due to automatic review settings August 13, 2026 02:15

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 02:15 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/pkg/dataset_api/operations.go

Copilot AI review requested due to automatic review settings August 13, 2026 02:24

Copilot started reviewing on behalf of Mohamed Hessien (m7md7sien) August 13, 2026 02:25 View session

Copilot AI reviewed Aug 13, 2026

View reviewed changes

Comment thread cli/azd/extensions/azure.ai.dataset/internal/cmd/dataset.go

Navigation Menu

Uh oh!

Add azd ai dataset extension for Foundry datasets - #9499

Add azd ai dataset extension for Foundry datasets#9499
Mohamed Hessien (m7md7sien) wants to merge 30 commits into
Azure:mainfrom
m7md7sien:feat/azure-ai-dataset

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 •

edited

Loading

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		var respErr *azcore.ResponseError
		if !errors.As(err, &respErr) {

		var respErr *azcore.ResponseError
		if errors.As(err, &respErr) {

		cmd.Flags().StringVar(&version, "version", "",
		"Current version to increment from. Omit to increment from the latest registered version.")

Uh oh!

Conversation

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why its own extension

Surface

Notable

Verified

Not included, deliberately

Open question for reviewers

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

azure-pipelines Bot commented Aug 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Mohamed Hessien (m7md7sien) commented Aug 10, 2026 •

edited

Loading