Skip to content

feat: unify agent environment diagnostics - #9531

Merged
Hui Miao (huimiu) merged 6 commits into
mainfrom
hui/unified-env-diagnostics
Aug 13, 2026
Merged

feat: unify agent environment diagnostics#9531
Hui Miao (huimiu) merged 6 commits into
mainfrom
hui/unified-env-diagnostics

Conversation

@huimiu

@huimiu Hui Miao (huimiu) commented Aug 11, 2026

Copy link
Copy Markdown
Member

Why this is needed

Issue #8710 still had inconsistent environment diagnostics: next-step guidance and azd ai agent doctor read legacy agent.yaml directly, while current projects can define the same values inline in azure.yaml or under deprecated config:. Users could therefore receive different or incomplete remediation guidance depending on the configuration shape.

What this PR does

  • Loads effective agent environment values with precedence for service-level env:, inline agent properties, deprecated config:, and legacy agent.yaml/agent.yml fallback.
  • Reuses those values for next-step missing-variable and placeholder detection.
  • Records environment configuration load errors in shared state and reports them from Doctor instead of silently passing.
  • Keeps existing legacy fallbacks and limits the scope to environment diagnostics; model, connection, toolbox, protocol, and deployment ownership migration are not included.

E2E validation

Command Result
azd ai agent doctor --local-only Pass for inline azure.yaml environment values.
azd ai agent doctor --local-only Pass for a local $ref environment source.
azd ai agent doctor --local-only Pass for deprecated config: environment values.
azd ai agent doctor --local-only Pass for legacy agent.yaml environment values.
azd ai agent init --no-prompt -m agent.manifest.yaml Pass from an empty directory with isolated authentication; unified Next guidance was rendered.

The negative diagnostic scenarios also produced actionable azd env set guidance for missing variables and source-neutral configuration repair guidance for a malformed environment reference.

Command Result
azd provision --no-prompt Pass
azd deploy --no-prompt Pass
azd ai agent run Pass

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
21 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@huimiu Hui Miao (huimiu) linked an issue Aug 11, 2026 that may be closed by this pull request
@huimiu
Hui Miao (huimiu) marked this pull request as ready for review August 11, 2026 07:14
Copilot AI balanced review requested due to automatic review settings August 11, 2026 07:14
@github-actions

Copy link
Copy Markdown

📋 Prioritization Note

Thanks for the contribution! The linked issue isn't in the current milestone yet.
Thank you for logging this issue; our team is reviewing it. If you need urgent prioritization, tag Rick Winter (@RickWinter) and Kristen Womack (@kristenwomack) to let us know.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
21 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Unifies agent environment diagnostics across unified, deprecated, referenced, and legacy configuration shapes.

Changes:

  • Loads effective environment values using the intended precedence.
  • Reuses them for missing-variable and placeholder diagnostics.
  • Reports environment-loading failures through Doctor.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
nextstep/types.go Extends diagnostic state with environment data and errors.
nextstep/state.go Collects effective values and detects missing references.
nextstep/state_test.go Tests unified and referenced configurations.
nextstep/resolver.go Generalizes remediation wording.
nextstep/resolver_test.go Updates resolver expectations.
nextstep/format_test.go Updates formatted-output expectations.
nextstep/environment.go Implements effective environment loading.
nextstep/environment_test.go Tests precedence, fallbacks, and errors.
doctor/checks_manual_env.go Reports environment-loading failures.
doctor/checks_manual_env_test.go Tests Doctor failure reporting.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Copilot AI review requested due to automatic review settings August 11, 2026 08:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (2)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go:1106

  • [azd-code-reviewer] environmentValuesFromManifest silently converts malformed YAML into an empty environment. Cases that expect no refs/placeholders would then pass for the wrong reason; fail the test immediately on fixture parse errors.
	var hosted agent_yaml.ContainerAgent
	if err := yaml.Unmarshal([]byte(data), &hosted); err != nil {
		return nil
	}

cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_manual_env.go:124

  • [azd-code-reviewer] This loader also supports azure.yml, so directing every affected user to fix azure.yaml can point them at a file that does not exist. Refer to the project configuration file generically (or preserve its actual path) in the remediation.
					Suggestion: "Fix azure.yaml, its referenced file, or the legacy agent file, then re-run `azd ai agent doctor`.",

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go Outdated
Copilot AI review requested due to automatic review settings August 12, 2026 03:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Suppressed comments (5)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go:1285

  • Use t.Context() here so cancellation follows the test lifecycle, as required by the repository's Go test pattern.
	state, errs := assembleState(context.Background(), src)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go:1318

  • Use t.Context() here so cancellation follows the test lifecycle, as required by the repository's Go test pattern.
	state, errs := assembleState(context.Background(), src)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go:1072

  • The leading indentation makes this YAML fixture malformed, and environmentValuesFromManifest silently converts that parse failure to an empty input. The test therefore passes without exercising either Foundry expression. Remove the extra indentation so this case validates the placeholder filtering behavior.
 environment_variables:

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state_test.go:1245

  • Use t.Context() here so cancellation follows the test lifecycle, as required by the repository's Go test pattern.

This issue also appears in the following locations of the same file:

  • line 1285
  • line 1318
	state, errs := assembleState(context.Background(), src)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/environment.go:82

  • LoadServiceEnvironment resolves the raw service entry without the runtime loader's validateRootRefCoreFields check (internal/project/agent_definition.go:584-613). Consequently, an env block supplied by a root $ref is treated as effective here and can generate missing-variable guidance, while run and deploy reject that same configuration. Route diagnostics through shared resolution/validation semantics (or extract them into a lower-level package) before merging the environment maps.
	serviceEnvironment, err := projectconfig.LoadServiceEnvironment(
		projectRoot,
		svc.GetName(),
	)
	if err != nil {

Comment thread cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_manual_env.go Outdated
Copilot AI review requested due to automatic review settings August 12, 2026 04:10

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (2)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/resolver.go:164

  • azd-code-reviewer: Environment load failures are not treated as blockers in this resolver. collectServices now records EnvironmentLoadErrors while leaving that service's values empty, so a malformed env or $ref can make this function reach the default branch and recommend azd ai agent run. Add source-neutral repair guidance and prevent runnable follow-ups until these errors are fixed.
	// Placeholder fix-ups come first when present: they are
	// broken state in the agent configuration and block both `run` and
	// `deploy`. The user has to edit the agent configuration (or define a
	// matching parameter in agent.manifest.yaml) — `azd env set` cannot
	// reach them.

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go:735

  • azd-code-reviewer: This special case breaks the documented lockstep with agent_yaml.ExtractUnresolvedPlaceholders. The init warning still treats ${{project.endpoint}} and $${{project.name}} as unresolved and tells users to replace them, while Next now suppresses them. Move Foundry-expression filtering into the shared placeholder extractor so both diagnostics agree.
		for _, m := range placeholderPattern.FindAllStringSubmatchIndex(value, -1) {
			if len(m) < 4 || (m[0] > 0 && value[m[0]-1] == '$') {
				continue

@trangevi

Copy link
Copy Markdown
Member

Copilot resolve the merge conflicts in this pull request

…ostics

# Conflicts:
#	cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go

Co-authored-by: trangevi <26490000+trangevi@users.noreply.github.com>
Copilot AI review requested due to automatic review settings August 12, 2026 17:53

Copilot AI commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Copilot resolve the merge conflicts in this pull request

Resolved in 05a64d4 by merging origin/main and combining the EnvironmentValues field with the isVoiceService parameter in collectServices.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Suppressed comments (3)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go:673

  • [azd-code-reviewer] This now treats every ${NAME} found in the merged values as an azd-environment dependency, but service-level env keys can satisfy aliases directly. For example, inline TARGET: ${SERVICE_ENDPOINT} plus service env.SERVICE_ENDPOINT works through ResolveAgentEnvironmentVariable (internal/project/agent_definition.go:274-277), while this path asks src.EnvValue for SERVICE_ENDPOINT and incorrectly reports it missing. Preserve the service-level environment keys per service and exclude references satisfied by those keys before classifying missing variables.
		refs, phs := extractEnvironmentRefs(svc.EnvironmentValues)

cli/azd/extensions/azure.ai.agents/internal/cmd/nextstep/state.go:753

  • [azd-code-reviewer] Filtering Foundry expressions only here breaks the package's explicit lockstep invariant: agent_yaml.ExtractUnresolvedPlaceholders still applies PlaceholderPattern without this exclusion (internal/pkg/agents/agent_yaml/placeholders.go:45), and parameters.go:81-84 requires its warning and Next guidance to agree. A valid ${{project.endpoint}} can therefore still trigger the init warning while disappearing from Next guidance. Move the Foundry-aware filtering into the shared placeholder extractor and use it at both call sites.
		for _, m := range placeholderPattern.FindAllStringSubmatchIndex(value, -1) {
			if len(m) < 4 || (m[0] > 0 && value[m[0]-1] == '$') {
				continue

cli/azd/extensions/azure.ai.agents/internal/cmd/doctor/checks_manual_env.go:125

  • [azd-code-reviewer] This remediation names azure.yaml, but the loader also supports azure.yml; users of that supported filename are directed to a file that may not exist. Keep the suggestion source-neutral, consistent with the new diagnostic.
					Suggestion: "Fix azure.yaml, its referenced file, or the legacy agent file, " +
						"then re-run `azd ai agent doctor`.",

@huimiu

Copy link
Copy Markdown
Member Author

/check-enforcer override

@huimiu
Hui Miao (huimiu) enabled auto-merge (squash) August 13, 2026 02:09
@huimiu
Hui Miao (huimiu) merged commit cc24659 into main Aug 13, 2026
30 checks passed
@huimiu
Hui Miao (huimiu) deleted the hui/unified-env-diagnostics branch August 13, 2026 02:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ext-agents azure.ai.agents extension

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unify agent environment diagnostics

5 participants