Skip to content

docs: add Pipeline Policy Constraints with Tekton and Kyverno solution - #830

Open
l-qing wants to merge 8 commits into
mainfrom
docs/pipeline-policy-constraints-tekton-kyverno
Open

docs: add Pipeline Policy Constraints with Tekton and Kyverno solution#830
l-qing wants to merge 8 commits into
mainfrom
docs/pipeline-policy-constraints-tekton-kyverno

Conversation

@l-qing

@l-qing l-qing commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

What

Adds a Solution-kind article:

  • docs/en/solutions/Pipeline_Policy_Constraints_with_Tekton_and_Kyverno.md

It covers admission-time policy constraints for Tekton pipelines with Kyverno on ACP: template/Task identity locking, gate-parameter effective-value validation, source/target restrictions, result-driven cancellation, scoped exemptions (PolicyException), and rollout/upgrade operations.

Version anchoring

Verified against ACP 4.3 with Alauda DevOps Pipelines v4.14.x, Kyverno v1.15.9-v4.3.2, and Alauda Artifact Hub Shim v1.0.0; the applicability matrix is declared in the document front section, ProductsVersion: 4.3.x in the frontmatter.

How this was verified

Rather than reviewed on paper, the material was executed against a live ACP 4.3 cluster:

  • Each policy and run manifest is extracted from this document (bound to the section it belongs to, so an edit that moves a block is caught rather than silently extracting the wrong one), applied to the cluster, and the expectations that section states are then asserted one at a time — allow/deny/skip verdicts, run terminal states and reasons, PolicyReport contents, and the exact shape of a cancellation.
  • Sections whose claims are about runtime behaviour, not admission alone, were run as real PipelineRuns: the gate-parameter contract and its two cancellation alternatives (§4.2.1-§4.2.3), the protected-branch gate profile (§4.2.4), result-driven cancellation (§4.6), and the PolicyException walkthrough (§5.3).
  • The §5.3 walkthrough was re-executed end to end on 2026-08-11 with all six steps behaving as documented: RBAC refuses the business identity's exemption, admission refuses its run in the exempt namespace, the violating run without an exemption ends CreateRunFailed naming the policy, the approved exemption makes the identical run succeed, the same config in a normal namespace still fails, and deleting the exemption restores the refusal.
  • Copy-paste scope, as the document's front section states: the policy assets and run manifests are the copy-paste-ready artifacts; probe commands are assembled by the reader from the §3.4 skeletons plus each section's expectation tables — they are not shipped as ready-made scripts.
  • The verification scripts and their logs are deliberately not part of this repository; they stay with the author. Ask if you want the log for a specific section.
  • After the live pass, the document went through several additional full review rounds (including an adversarial blind review and an en-zh parity cross-review); the fixes are in the follow-up commits on this branch.

Known environment dependency worth calling out for reviewers: the Hub coordinates used in §4.2.4/§4.5.1 resolve through the cluster's in-cluster Artifact Hub Shim. On a cluster whose hubresolver-config still points at the public artifacthub.io, those references cannot resolve; the endpoint is set through TektonConfig.spec.pipeline.hub-resolver-config (the ConfigMap itself is operator-owned).

Notes for reviewers

  • The companion article Software_Supply_Chain_Security_of_Alauda_Container_Platform_with_Tekton_and_Kyverno.md is referenced as a same-directory relative link; it exists in the en tree.
  • The id frontmatter field is left for the add_id.sh flow to assign.
  • Line en:5970 contains a deliberately fake, format-valid Slack token: it is the fixture for the trivy scanners=[secret] demonstration in §4.4 and must stay format-valid for trivy to detect it. Push protection was resolved via the used-in-tests allowance.

@l-qing
l-qing deployed to translate August 11, 2026 07:59 — with GitHub Actions Active
@l-qing
l-qing deployed to translate August 11, 2026 08:05 — with GitHub Actions Active
@l-qing
l-qing deployed to translate August 11, 2026 08:24 — with GitHub Actions Active
@l-qing
l-qing force-pushed the docs/pipeline-policy-constraints-tekton-kyverno branch from afe55c1 to 99586ca Compare August 11, 2026 08:27
@l-qing
l-qing deployed to translate August 11, 2026 08:28 — with GitHub Actions Active
Add a Solution-kind article at
docs/en/solutions/Pipeline_Policy_Constraints_with_Tekton_and_Kyverno.md
covering admission-time policy constraints for Tekton pipelines with
Kyverno on ACP: template/Task identity locking, gate-parameter
effective-value validation, source/target restrictions, result-driven
cancellation, scoped exemptions (PolicyException), and rollout/upgrade
operations.

Verified against ACP 4.3 with Alauda DevOps Pipelines v4.14.x, Kyverno
v1.15.9-v4.3.2, and Alauda Artifact Hub Shim v1.0.0; every code block
was validated on a live cluster. ProductsVersion is anchored to 4.3.x;
the id frontmatter field is left for the add_id.sh flow to assign.

Note: the document deliberately ships a fake, format-valid Slack token
as the fixture for the trivy secret-scanning demonstration; it must
stay format-valid for trivy to detect it.
@l-qing
l-qing force-pushed the docs/pipeline-policy-constraints-tekton-kyverno branch from 99586ca to 2eb6d9c Compare August 11, 2026 08:33
@l-qing
l-qing deployed to translate August 11, 2026 08:33 — with GitHub Actions Active
@l-qing l-qing changed the title docs: add Pipeline Policy Constraints with Tekton and Kyverno solution (zh+en) docs: add Pipeline Policy Constraints with Tekton and Kyverno solution Aug 11, 2026

@yuzichen12123 yuzichen12123 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

本次结合 DEVOPS-43870 及其关联文档单 DEVOPS-43674 的验收标准完成审查,共提交 4 条行级评论:2 个 Major、2 个 Minor。重点是修正文档对 mutate-existing 故障语义的安全承诺,并补齐可直接执行、可重复验证的 probe 与测试证据。指定验收环境当前持续返回 EOF,待问题修复且环境恢复后,应补做目标版本的故障注入与完整正/负回归。


| What to budget | Mechanism facts | The budget / action you must set |
|---|---|---|
| **External calls on the admission path** | Two kinds of criteria wait for an external round trip inside admission: `context.imageRegistry` ([§4.5.2](#s4-5-2)) and `context.apiCall` (one each in [§4.2.1](#s4-2-1) / [§4.2.2](#s4-2-2) / [§4.6.1](#s4-6-1) — three in the whole document). **Both fail closed**: registry unreachable → request denied (measured at roughly 5 seconds; roughly 3 seconds when reachable, see limitation 4 in [§4.5.2](#s4-5-2)); apiCall cannot fetch its target → the rule errors out and the request is denied (the error message shape is in the [§4.2.1](#s4-2-1) warning) | Decide explicitly "which request paths may carry external calls"; narrow such rules' match down to **the Tasks that genuinely need them**; load-test p95 / p99 and the timeout ratio, and confirm it stays below the webhook timeout — **that ceiling applies to the whole request, not to each rule separately** (measured `timeoutSeconds=10` in this document's verification environment, see checklist item 6 in [§3.1](#s3-1); one 5-second registry round trip fits, two stacked on the same request may not). **Registry / API server jitter turns directly into pipeline creation failures** — have the matching alerts and playbooks ready |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|mutate-existing 的 apiCall 故障语义被误写为 fail-closed

这里把 §4.2.1、§4.2.2、§4.6.1 的三个 context.apiCall 一概描述成“查询失败时 rule error,原请求被拒绝”。这只适用于同步 validate 路径,不能套用到 mutate-existing。

Kyverno 1.15 上游实现中,mutate-existing 由 handleBackgroundApplies 异步处理,并且只为 RuleStatusPass 创建 UpdateRequest;context/precondition 得到 RuleStatusError 时不会进入 admission 的 BlockRequest。因此 §4.2.2/§4.6.1 查询父 PipelineRun 失败时,原 TaskRun CREATE 或 status UPDATE 仍可能被接受,但预期的 CancelledRunFinally 补丁不会产生。此时是 fail-open,流水线可能继续执行。

修复目标:

  1. 分开描述同步 validate(§4.2.1)与异步 mutate-existing(§4.2.2/§4.6.1)的错误语义,不再使用统一的 “Both fail closed”。
  2. 明确 mutate-existing 在 apiCall、规则求值或 UpdateRequest 创建失败时不会完成取消,并补充 UpdateRequest、background-controller event/error 的监控与告警要求。
  3. 说明需要零竞态硬保证时应使用同步 validate/admission-mutate,而不能把异步取消当作硬门禁。
  4. 在目标 Kyverno v1.15.9-v4.3.2 环境补一条受控故障用例:制造父对象查询失败,验证原请求、取消补丁和告警三者的实际结果,并据此更新文字。

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

补充目标环境实测证据(2026-08-11,Tekton Pipelines v1.12.2,Kyverno v1.15.9-v4.3.2):

  • 正常对照组:mutate-existing 规则的 context.apiCall 查询当前真实 PipelineRun,并校验返回 UID;PipelineRun 正常完成,目标注解成功写入,证明规则、RBAC 和 background mutation 路径本身可用。
  • 故障注入组:保持 match、subject、target、RBAC 和触发方式不变,仅将 apiCall 指向同命名空间内一个确定不存在的 PipelineRun。结果是原 PipelineRun 的 status UPDATE 未被拒绝,最终状态为 Succeeded;预期 mutation 注解未写入;查询时没有保留该策略对应的 UpdateRequest,admission/background controller 日志中也没有观察到包含该测试策略名的错误记录。
  • 因此差异可以归因于 apiCall 故障:在 mutate-existing 路径中,实际行为是“原请求继续、mutation 消失”,不是本行描述的 fail-closed。且本次未自然产生可操作告警,文档还需明确补充监控手段,不能只要求查看 background-controller event/error。

这也进一步确认修复目标 1–4 均为必须项,尤其应把“异步取消失败”作为显式验收用例。

@l-qing l-qing Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 c966ff7 修复——也感谢你补充的受控故障注入实测,与机制推演完全吻合。

文档不再用一句 "both fail closed" 概括三处 apiCall

  • §3.7 拆成两行:「准入路径上的外部调用(同步 validate)」——只有 §4.2.1(以及 §4.5.2 的 imageRegistry 检查)是 fail-closed;新增一行**「异步投递链路(mutate-existing 取消是 fail-open)」**覆盖 §4.2.2 / §4.6.1 / §4.6.2:该链路上任何一环失败(apiCall 查询、UpdateRequest 创建、background-controller 可用性、目标对象 update RBAC)都会放行原请求,取消补丁静默消失。
  • 同一行明确**「判据方向 fail-closed ≠ 交付保证」**,并列出必须的监控项:background-controller 存活/积压、UpdateRequest 创建与失败量、策略命中数与实际打上补丁对象数的差值,以及上线前和每次升级后的受控故障注入演练(正是你复现的场景)。
  • §2 观察点表与 §4.6.1 带上了同样的限定;文档现在明确写出:零竞态硬保证必须走同步路径(§4.2.1 deny / §4.2.3 admission mutate),异步取消不能当硬门禁使用。


Later sections supply probes in one of two ways: some give complete manifests and commands directly ([§3.3](#s3-3), [§4.4.1](#s4-4-1), [§4.4.2](#s4-4-2), [§4.6.1](#s4-6-1)); the others give only an **expectation table** (listing which inputs should be allowed / denied / skipped) and leave the commands to this section — because these three kinds of commands are mechanical, and repeating them section by section would make the document longer, not clearer.

**Eight sections give only an expectation table and need you to turn it into commands yourself**: [§4.1.1](#s4-1-1), [§4.2.4](#s4-2-4), [§4.5.1](#s4-5-1), [§4.5.3](#s4-5-3), [§4.5.4](#s4-5-4), [§4.5.5](#s4-5-5), [§5.2](#s5-2), [§5.3](#s5-3). (The method behind this list is "the section has an expectation table but no `kubectl create` that submits a probe" — after the document changes, recount by that method instead of trusting the list. The `kubectl apply` commands already present in [§4.5.3](#s4-5-3), [§5.2](#s5-2), [§5.3](#s5-3) create **prerequisite objects** (ConfigMaps, namespaces), not the probes themselves.) In addition, [§4.1.4](#s4-1-4) and [§4.1.5](#s4-1-5) have no expectation table: they are Audit defense in depth on `*/status`, their criteria are stated in the body, and the expected shape is "which fail entry should appear in the report" — follow **type 2** below.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|缺少可直接执行的 probe 和可复核的 731 条断言产物

本节明确说明有 8 个章节只有 expectation table,需要读者自行选择 skeleton、编辑字段并生成 probe.yaml。受影响的章节包括 §4.1.1、§4.2.4、§4.5.1、§4.5.3、§4.5.4、§4.5.5、§5.2、§5.3。这与前文对 walkthrough operator 的 “Every step copy-pasteable” 承诺以及 Jira 的“所有示例 YAML 可直接执行并验证通过”验收标准不一致。

另外,PR 描述声称已完成 731 条 live assertions,但当前变更只包含 Markdown,没有提交或链接脱敏后的 harness、输入版本、执行日志/结果摘要以及“策略/探针 → 断言”映射。现有 GitHub build 只能证明站点可以构建,不能证明策略行为和清理流程已经回归。

修复目标:

  1. 为上述 8 个章节补齐完整的正/负 probe manifest,可集中成多文档 YAML,但必须能在替换环境参数后直接提交。
  2. 每个 probe 提供实际执行命令、机器可判定的预期结果和清理命令,避免依赖读者自行解释输出。
  3. 将脱敏后的测试 harness 纳入仓库,或提供稳定、可访问的结果产物;记录 ACP、Pipelines、Kyverno、Artifact Hub Shim 的精确版本。
  4. 建立每个完整策略及 probe 到断言结果的映射,使升级后可以重复回归。若本次只计划交付设计矩阵,则应删除 copy-paste/已全部验证的承诺,并同步调整验收范围。

@l-qing l-qing Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 c966ff7 部分采纳,并说明一下范围口径:

  • "Every step copy-pasteable" 的承诺已删除。读者画像行现在明确写:策略与运行清单可直接复制;九个小节的探针按 §3.4.1 拼装——该节现在给出通用配方(三类探针的具体命令、身份参数与取证路径)加三份现成骨架 manifest,并逐一点名这九个小节。
  • 关于 harness 与日志:我们有意不把过程性产物(测试 harness、执行日志、逐条断言结果)放进本仓库——这是文档仓库,不适合承载过程数据。PR 描述里的断言数量是内部验证的背景说明,不是本 PR 的交付物。
  • 也说明一下与此选择相关的定位:本 KB 的目标是讲清底层机制(准入阶段、resolver 行为、mutate-existing 投递语义、判据设计)。具体的 Task/Pipeline 定义因环境和用户而异——读者需要按自己的契约改判据,这正是若干小节刻意提供期望表 + 骨架、而非钉死环境的探针的原因。


**The key criterion** — each of the three channels locks the complete identity, and a false union is a denial:

```yaml

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor|这些 yaml fence 是非完整片段,无法独立解析

这里及另外 9 处 “Key criterion” 代码块只截取了嵌套字段,缺少外围对象。逐块运行 yq eval 时共有 10/69 个 yaml fence 解析失败,起始行分别为 2140、2434、2637、2886、3577、4413、5337、5564、6052、7675。后续完整策略 YAML 可以解析,因此这不是策略本体错误,但当前语言标记容易让读者和自动化把它们当成可复制的 YAML 示例,也与“所有示例 YAML 可直接执行”的表述冲突。

修复目标:

  • 如果只用于解释逻辑,将这 10 个 fence 改为 text,并在标题或块前明确标注 non-executable YAML fragment
  • 如果希望保留 yaml,则补齐到可独立解析的最小对象。
  • 修复后以“所有标为 yaml 的 fence 均能逐块通过 YAML 解析”作为验收条件。

@l-qing l-qing Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 c966ff7 通过标注处理:这 10 个「关键判定」块是刻意的节选——用来讲解每条策略里起决定作用的那几行,完整可安装的 YAML 一定在同节的 details 块里。10 处现在都带了行内标注:"(fragment, not a complete manifest you can kubectl apply as-is; the full policy is in this section's details block)"。保留 yaml 标记是为了语法高亮;有了标注就不存在复制歧义,且"可复制"承诺现在只覆盖策略与运行清单(见探针那条评论的回复)。


Requiring merely that "a task named `scan` exists in the inline definition" is an **insufficient** security check — an attacker can drop in a hollow no-op scanner, hang a never-true `when` on it so it gets skipped, set the gate switch to `false`, or order release before it / parallel to it.

To open up inlining safely, you must validate the complete contract set of [§2.3](#s2-3) (scanner identity, effective values of the gate switches, must-run, DAG d

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor|4.14 安装链接当前跳转到了 4.12 首页

当前 https://docs.alauda.io/alauda-devops-pipelines/4.14/install.html 返回的 HTML 使用 meta refresh 跳转到 /alauda-devops-pipelines/4.12/,并且丢失了 install.html 路径。读者按这里查找 4.14 的安装和 TektonConfig 说明时,实际看到的是错误版本、错误页面,无法核对本文声明的 4.14 前提。

修复目标:

  1. 替换为实际存在的 4.14 安装/TektonConfig 文档地址。
  2. 如果 4.14 文档尚未发布,明确说明临时引用的版本、与 4.14 的差异及后续替换条件,不能把 4.12 首页标成 4.14 安装页。
  3. 验收时检查最终页面内容与版本,而不只检查 HTTP 200;当前页面虽然返回 200,但正文会继续重定向。

@l-qing l-qing Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

作为已知问题处理,修复在途:Alauda DevOps Pipelines 的 4.14 文档即将上线,届时该链接会解析到正确版本。我们保留 4.14 URL 不动,让它在文档发布后自然生效,而不是把读者永久指到 4.12。

kubectl get cm -n "$TEKTON_NS" resolvers-feature-flags -o jsonpath='{.data}{"\n"}'
# Expect enable-cluster-resolver / enable-hub-resolver / enable-git-resolver to be
# "true" as required by the resolvers you actually use
echo "hub default-type: $(kubectl get cm -n "$TEKTON_NS" hubresolver-config -o jsonpath='{.data.default-type}')"

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|Hub 检查会误报就绪,文档中的官方模板在验收环境实际无法解析

在提供的 /root/config/kb.yaml 环境实测:

  • enable-hub-resolver=true,本行读取的 default-type=artifact,因此当前检查会判定 Hub 能力正常;
  • 按文档完整坐标创建 java-image-build-scan-deploy@0.3python-image-build-scan-deploy@0.3 的真实 PipelineRun,二者均终止为 CouldntGetPipeline,resolver 请求返回 404,status.pipelineSpec 未生成;显式 type=artifact 也不会改变这个 endpoint;
  • 同一集群内置 Artifact Hub Shim 对这两个 Pipeline 以及 sonarqube-scanner@0.7trivy-scanner@0.6skopeo-copy@0.1 的精确坐标均返回 200,解析 manifest 后参数、结果和 DAG 合约也与 §3.2 矩阵一致。问题是当前 resolver 配置的 endpoint 没有实际提供这套内置 catalog,而现有检查完全发现不了。

这会使所有依赖真实 Hub profile 的示例在 walkthrough 开始很久以后才以解析失败结束,用户容易误判为策略问题,也不满足“示例可直接执行验证”的验收目标。

修复目标:

  1. 这里不能只检查 feature flag 和 default-type;增加对当前配置的 artifact-hub-api endpoint 的真实 smoke test,至少验证本文固定使用的五个 catalog/name/version/kind 坐标可解析。
  2. 提供可复制命令并机器判定:官方 Pipeline 应生成 status.pipelineSpec,Task 应生成已解析 spec;404、认证失败或 endpoint 不一致必须在进入 Cookbook 前明确失败。
  3. 明确“内置 Shim”与 resolver 实际配置 endpoint 的关系;若验收环境配置漂移,应先修复环境再宣称本文 live assertions 通过。
  4. 把该 smoke test 纳入可重复 harness,避免仅验证 Shim 自身“有包”,却漏掉 resolver 实际“取不到包”的链路断点。

@l-qing l-qing Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

已在 c966ff7 修复——§3.1 核查第 2 项不再只信 feature flags:

  • 新增 2b 冒烟:从工作负载 Pod 直接探测实际配置的 artifact-hub-api endpoint,用精确版本详情接口(.../api/v1/packages/<type>/<catalog>/<name>/<exact-version>)核对本文钉死的五个坐标(sonarqube-scanner/0.7trivy-scanner/0.6skopeo-copy/0.1java-image-build-scan-deploy/0.3python-image-build-scan-deploy/0.3)。
  • 通过标准机器可判定且严于 HTTP 200:每个坐标必须返回 200 data.manifestRaw 非空(包列表 200 不够——钉住的版本或其 manifest 仍可能缺失);任一坐标失败整块非零退出,问题在进入 Cookbook 之前就暴露,而不是很晚才以 CouldntGetPipeline 的形式出现。
  • 通过标准表要求五个坐标全部可用,分诊列区分你遇到的两类失败:404 → endpoint/catalog 漂移(含 flags 全绿但 endpoint 指错的情况,按环境问题先修再继续);UNREACHABLE → 工作负载 Pod 的出口/DNS 不通。

- Split failure semantics of synchronous validate vs asynchronous
  mutate-existing paths: the latter is fail-open (request admitted,
  cancellation patch silently lost); add delivery-chain monitoring and
  fault-injection requirements, and state that hard guarantees need the
  synchronous paths
- Harden the Hub readiness check: probe the five pinned exact-version
  coordinates against the configured artifact-hub-api endpoint and
  require HTTP 200 plus non-empty data.manifestRaw
- Label the ten "Key criterion" excerpts as non-executable fragments;
  scope the copy-paste promise to policies and run manifests, with the
  probe recipe and skeletons consolidated in §3.4.1
- Rework demo cleanup to a UID ownership ledger for cluster-scoped
  objects instead of delete-by-name
- Assorted criterion, probe-table, and placeholder-guidance refinements
  across §2, §3, §4 and §5
@l-qing
l-qing deployed to translate August 12, 2026 13:52 — with GitHub Actions Active
Port the corrections found by re-verifying the document against a live
Kyverno v1.15.9 / Tekton cluster:

- 4.4.1: the prose listed the 6th and 7th expected verdicts in the wrong
  order, and claimed the counting form and the [0] form are "identical"
  without saying on what. Scope it to the seven normal shapes and name the
  forged-condition construction as the only divergence.
- 2.3 / 4.2.1: "no retry" means the run will not retry forever, not that the
  controller attempts creation exactly once; a single run's
  TaskRunsCreationFailed event can carry count > 1. Add the per-run counting
  command so the reader can check it.
- 2.1 row 8: an apiCall's failure direction depends on the rule it sits on -
  fail-closed on a synchronous validate rule, fail-open on mutate-existing.
- 6.2.3 row 4: fail-closed is the direction of the criterion, not a delivery
  guarantee.
- 4.5.2: build and org.opencontainers.image.source may exist only one at a
  time; document the earlier failure point where a private or link-local
  www-authenticate realm is refused before authentication is attempted.
- 4.6.1: a cancellation landing inside the child TaskRun's initialization
  window also ends in Failed, unrelated to whether the result was written.
- Quote four bare <placeholder> occurrences in bash fences: unquoted < and >
  are shell redirections that silently truncate the command (also clears
  four SC2261 findings).
- Rewrite six author-process passages into reader-facing statements.
@l-qing
l-qing deployed to translate August 13, 2026 03:29 — with GitHub Actions Active
Follow-up to the previous commit, from an independent review of it:

- 6.2.3 said four paths "produce Cancelled" while the same paragraph and
  4.6.1 both document cancellations that end in Failed. Say the paths
  request a cancellation and point at the terminal state that may differ.
- 2.1 row 8 narrowed a failed apiCall to "a lookup that comes back empty",
  which drops the unreachable and forbidden cases 4.2.1 explicitly covers.
- The factory resourceFilters set and the built-in hub's catalog name were
  stated as product-wide facts; both are defaults to verify against the
  cluster, and each sentence already carries the command that reads them.
@l-qing
l-qing deployed to translate August 13, 2026 03:36 — with GitHub Actions Active
| 3 mutate-existing RBAC | Returns `yes` if you use the [§4.6](#s4-6) cancellation capability | On `no`, grant the aggregated ClusterRole given in the [§4.6](#s4-6) preamble (the `rbac.kyverno.io/aggregate-to-background-controller: "true"` label in its labels aggregates it into the background controller's permissions). **If you want to use a namespaced Role instead, you must also change `mutate.targets[].namespace` from `{{ request.namespace }}` to a namespace literal** — otherwise Kyverno's creation-time authorization check cannot resolve that variable, recognizes only cluster-level permissions, and the policy still fails to install (see the [§4.6](#s4-6) preamble). **If you do not install the [§4.6](#s4-6) cancellation policies, this permission is not needed** |
| 4 reports-controller reads status | All six `yes` (optional, not required) | A `no` **usually needs no action** (rationale in the third interpretation below). Only when some other feature genuinely needs the reports-controller to read status directly, add one more least-privilege ClusterRole the same aggregated way as item 3, with the aggregation label swapped to `rbac.kyverno.io/aggregate-to-reports-controller: "true"` |
| 5 PolicyException switches | Both `--enablePolicyException=true` and `--exceptionNamespace=<trusted-namespace>` present | Seeing only the former is ACP's default state — per [§3.1.1](#s3-1-1), write the `enabled` / `namespace` of `features.policyExceptions` into the kyverno `ModuleInfo`'s `spec.valuesOverride["ait/chart-kyverno"]` (**`ModuleInfo` exists only on the global management cluster**, see the warning in [§3.1.1](#s3-1-1)); **do not patch the Deployment args**. [§3.1.1](#s3-1-1) provides copy-pasteable atomic patch and rollback commands. **If you do not plan to use PolicyException exemptions ([§5.3](#s5-3)), you need not configure this** |
| 6 Webhook failure policy and timeout | **No fixed expected value** — it is planning input, not a pass/fail criterion. ⚠️ **The reading is timing-sensitive**: `kyverno-resource-validating-webhook-cfg` (the one that actually governs `PipelineRun` / `TaskRun` / `Pod`) is **generated dynamically by Kyverno from the installed policies** — with none of this document's policies installed its `webhooks` is empty and this line prints `<none>`. The `Fail/10` lines you can read at that point all belong to the webhooks of Kyverno's **own CRs** (policy / exception / cleanup / ttl). To get the actual values for pipeline resources, **come back and read this line after installing any [§4](#s4) policy** | Record the actual values and build your playbook around them: `Fail` requires guaranteed controller replicas and HA ([§6.1.8](#s6-1-8)); `Ignore` means accepting a policy vacuum while Kyverno is unavailable. `timeoutSeconds` is the **total budget for a single request** (the default under the applicable versions is `failurePolicy=Fail` / `timeoutSeconds=10`); the external calls in [§3.7](#s3-7) must fit inside that number. If you genuinely need to adjust it, go through the [§3.1.1](#s3-1-1) entry point as well — **do not edit the `ValidatingWebhookConfiguration` directly**: it is an object Kyverno itself maintains (it carries `webhook.kyverno.io/managed-by=kyverno`) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|failurePolicy 的配置入口写错,后文要求的按策略分层实际无法落地

这里把生成后的 ValidatingWebhookConfiguration 当成主要观测点,并把调整入口导向 §3.1.1 的 ModuleInfo;但在本文适用的 Kyverno 1.15 中,经典 Policy 的策略级配置入口是 spec.webhookConfiguration.failurePolicy(旧 spec.failurePolicy 已废弃)。Kyverno CRD 明确说明:同一 Policy 内的规则共享该故障行为,允许 Ignore|Fail,默认 Fail。派生 webhook 只是 Kyverno 根据各 Policy 生成的生效结果,ModuleInfo/全局 forceFailurePolicyIgnore 也不能表达 §3.7 所要求的“硬门禁偏向 Fail、记账型 Audit 可选 Ignore”的逐策略分层。

我在更新后的验收环境(Kyverno 1.15.9-v4.3.2)用独立 namespaced Policy 设置 spec.webhookConfiguration.failurePolicy: Ignore 验证,生成的 resource webhook 确实进入 Ignore 分组;而本文 25 份 Policy YAML 均未声明 webhookConfiguration,因此当前示例全部落到默认 Fail,和后文的分层建议脱节。同步 apiCall / imageRegistry 的“错误时拒绝”结论也应明确以该 Policy 的 failurePolicy 为 Fail 为前提。

修复目标:

  1. 在每份完整 Policy 示例中显式声明 spec.webhookConfiguration.failurePolicy(以及需要定制时的 timeoutSeconds),让安全意图进入 GitOps 管理的策略本体;硬门禁及同步外部调用通常选 Fail,选择 Ignore 的 Audit 策略需明确接受的真空边界。
  2. 检查项先读取 Policy spec 的配置意图,再核对 Kyverno 生成的 webhook 是否按 Fail/Ignore/timeout 分组生效;不要只读派生 webhook。
  3. 说明 ModuleInfo/全局 override 只能用于平台级强制覆盖,不能替代 per-policy 配置;仍然不要直接编辑生成的 VWC。

| What to budget | Mechanism facts | The budget / action you must set |
|---|---|---|
| **External calls on the admission path (synchronous validate)** | Two kinds of criteria wait for an external round trip inside admission: `context.imageRegistry` ([§4.5.2](#s4-5-2)) and `context.apiCall` (one each in [§4.2.1](#s4-2-1) / [§4.2.2](#s4-2-2) / [§4.6.1](#s4-6-1)). **Only the two on the synchronous validate path fail closed**: registry unreachable → request denied (roughly 5 seconds; roughly 3 seconds when reachable, see limitation 4 in [§4.5.2](#s4-5-2)); the [§4.2.1](#s4-2-1) apiCall cannot fetch its target → the rule errors out and the request is denied (the error message shape is in that section's warning). **The apiCalls of [§4.2.2](#s4-2-2) / [§4.6.1](#s4-6-1) do not belong in this row** — they hang off mutate-existing and fail in the opposite direction; see the next row | Decide explicitly "which request paths may carry external calls"; narrow such rules' match down to **the Tasks that genuinely need them**; load-test p95 / p99 and the timeout ratio, and confirm it stays below the webhook timeout — **that ceiling applies to the whole request, not to each rule separately** (default `timeoutSeconds=10`, see checklist item 6 in [§3.1](#s3-1); one 5-second registry round trip fits, two stacked on the same request may not). **Registry / API server jitter turns directly into pipeline creation failures** — have the matching alerts and playbooks ready |
| **The asynchronous delivery chain (mutate-existing cancellation is fail-open)** | Of the document's four cancellation paths, three are mutate-existing ([§4.2.2](#s4-2-2) / [§4.6.1](#s4-6-1) / [§4.6.2](#s4-6-2)), and they sit **outside the admission verdict**: on a hit, the background-controller patches the target object asynchronously via an UpdateRequest. So when any link in that chain fails — the rule's `context.apiCall` cannot fetch its target, the UpdateRequest never gets created, the background-controller is down or backlogged, update RBAC on the target resource has been revoked — **the original request is allowed as usual and the cancellation patch silently vanishes**: the pipeline runs to the end, with no denial message anywhere in the cluster and no PolicyReport violation record (mutate types produce no violation records — the [§4.2.3](#s4-2-3) warning), only a few error lines in the background-controller log (the 404 note in [§4.6.1](#s4-6-1)). **criterion direction fail-closed ≠ delivery guarantee**: the criterion says "missing/illegal results must cancel just the same", but whether the cancel lands depends on the health of this background chain | **For a hard guarantee of zero races and zero silent failures, only the synchronous paths qualify**: the deny of [§4.2.1](#s4-2-1) or the admission mutate of [§4.2.3](#s4-2-3) (both give a synchronous verdict inside admission). If you stay on mutate-existing, you must monitor this chain as **a delivery system with an SLA**: ① background-controller liveness, restart count, and queue backlog; ② UpdateRequest creation volume and failure/stuck volume (`kubectl get updaterequests -n kyverno`); ③ the gap between policy hit counts and "target objects that really got `spec.status` / `cancel-reason` applied" — **a gap persistently non-zero means cancellations are being lost**; ④ before rollout and after every upgrade, run one controlled fault injection (point the apiCall at a nonexistent object, or temporarily stop the background-controller), confirm the alert fires, and write the result into the change request |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|异步取消的监控与故障注入方案在目标版本上不可直接执行

本行正确修复了 mutate-existing 的异步 fail-open 边界,但给出的运维方案还有两个可操作性问题:

  • 更新后的验收环境(Kyverno 1.15.9-v4.3.2)实测:制造唯一测试策略的 apiCall 404 后,原触发请求成功、目标补丁未落地,background-controller 有带策略名的错误;UpdateRequest 则快速经历 Pending → Failed、重试后立即删除。故障结束后普通 kubectl get updaterequests -n kyverno 返回 0 条,只有事先持续 watch 才能捕获。因此“creation volume / failure/stuck volume”不能仅靠这里给出的 get 命令建立,且“策略命中数”也没有说明可用的数据源。
  • “temporarily stop the background-controller”是集群级故障,会同时中断该集群其他 mutate-existing / generate 交付;本文又允许在 shared cluster 操作,不能把它作为“每次升级后”的无条件常规选项。

修复目标:

  1. 把可观测性写成可实施方案:给出持久化指标/日志告警的数据源,或明确要求在故障注入前启动 UpdateRequest watch/事件采集并把短生命周期状态外送;说明一次性 kubectl get 不能证明“无失败”。
  2. 定义“policy hit → target patch landed”差值的两端如何采集、关联(policy/rule、trigger UID、target UID/annotation)及告警窗口,避免留下无法计算的 SLA。
  3. shared/生产集群只使用唯一测试策略、测试命名空间和无害的不存在对象查询;若保留停 controller 的方式,限定为专用验收集群或维护窗口,并补充原副本状态记录、恢复、Ready 和积压恢复验证。

|---|---|---|
| 1 | The 7-item checklist of [§3.1](#s3-1) + the four configs `config-defaults` / `feature-flags` / `hubresolver-config` / the kyverno ConfigMap | Values match the pre-upgrade state; for any mismatch, first locate the affected policies per [§3.6](#s3-6) |
| 2 | The template allowlist of [§4.1.1](#s4-1-1) | The approved template is allowed; all three of an old version number, an unknown resolver, and a request-level `url` are denied |
| 3 | The gate parameter contract of [§4.2](#s4-2) (in whichever response shape you actually chose) | Compliant parameters are allowed; all three of switching the gate off / an explicit empty value / **an override field the criterion has never seen** are blocked (the third is the key probe for the denylist shape) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|升级回归要求旧 denylist 阻断“从未见过的字段”,验收目标与前文机制相反

前文 §3.6 已准确说明:denylist 只识别已知坏字段;新字段、新 override 入口会自动落到允许侧,升级时必须重新枚举字段面。这里却要求“criterion has never seen”的 override 字段已经被 blocked。两者不能同时成立:如果规则从未见过该入口,它按本文定义就不会阻断;只有先更新 criterion 把它纳入模型,负向探针才可能通过。

这会让升级验收产生错误判定:要么永远无法满足该行,要么测试人员把一个已知旧字段误称为“unknown”后得到虚假的回归通过。

修复目标: 将升级流程拆成两个明确阶段:

  1. 先以 API schema / kubectl explain / 真实对象 diff 发现新增字段,并用探针确认旧策略对它的实际方向;若允许,记录为待修缺口,不能判升级验收通过。
  2. 判断该字段是否能改变受保护行为;需要治理时先更新 allowlist/criterion,再验证更新后的策略拒绝该新入口,同时保留正常输入允许的正向用例。
  3. 将本行的通过标准改成“所有新增、且会影响受保护行为的 override 入口在策略更新后均被阻断”,不要要求“未见字段”天然被 denylist 阻断。

| Triggering action | Affected criteria | Consequence | What the change must include |
|---|---|---|---|
| Adding a workload namespace, or migrating pipelines to a new namespace | The `namespaces:` enumeration of **every** policy in [§4](#s4) (demo value `policy-poc`) | **Silent allow**: the new namespace matches no rule | First add the new namespace to every scope (or switch to namespaced `Policy` per [§5](#s5); for "covered by default" switch to a platform-level ClusterPolicy with a negative `exclude` for system namespaces, instead of enumerating one by one), run the positive/negative probes against the new namespace, and only **then** let the business move in |
| **Adding a workload cluster**, or migrating pipelines to another cluster | **Every policy in the whole document** ([§4](#s4) and [§5](#s5) alike) — `ClusterPolicy` / `Policy` are in-cluster objects and **do not sync across clusters** | **Silent allow**: the new cluster has not a single policy, while the old cluster's reports look perfectly fine — completely invisible from the old cluster | Write "install the minimal set ([§4.0.1](#s4-0-1)) + run the positive/negative probes" into the cluster onboarding process; distribute the policy inventory via GitOps / a platform module rather than installing by hand; periodically diff the `kubectl get clusterpolicy` inventories across clusters (listed as a lossy item in [§7.3](#s7-3)) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Major|多集群漂移检查只比较 ClusterPolicy,漏掉 namespaced Policy 和规则作用域

本行前半段已经明确“每个 ClusterPolicy / Policy 都是集群内对象”,但操作建议却只比较 kubectl get clusterpolicy。这既看不到项目自治使用的 namespaced Policy,也无法判断同名策略在不同集群上的 validate.failureActionmatch/excludenamespaceSelector 是否一致。后文 §7.3 又明确说比较不能停在策略名,至少要比较“name + 每条规则 failureAction + scope”,两处目前自相矛盾。

结果是两个集群即使 kubectl get clusterpolicy 名单相同,也可能一个仍为 Audit、另一个已 Enforce,或某个项目命名空间完全没有对应 Policy;此检查仍会误报基线一致。

修复目标:

  1. 将多集群基线定义为规范化后的 ClusterPolicy + 所有受管项目 namespaced Policy,而不是一条 kubectl get clusterpolicy 名单。
  2. 比较至少覆盖 policy namespace/name、每条 rule 的 validate.failureActionmatch/exclude 及 namespace selector;最好以 GitOps 期望态为基准做结构化 diff。
  3. 新集群 onboarding 与 §7.3 统一引用 §4.0.7 的完整适配和验收流程,避免这里只写“positive/negative probes”而遗漏身份、作用域和 Audit/cancellation 类验收。

@yuzichen12123 yuzichen12123 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

本轮按“机制说明文档”的定位复核最新提交 398e34f,并结合更新后的验收环境验证 Kyverno 1.15.9-v4.3.2 的 failurePolicy 与 mutate-existing/UpdateRequest 行为。已提交 4 条 Major 行级评论:逐策略 failurePolicy 配置入口、异步取消链的可观测性与安全故障注入、denylist 新字段的升级验收矛盾、多集群策略/作用域漂移检查。此前同步 validate 与异步 mutate-existing 故障语义、Hub 精确坐标 smoke test、片段标注等问题已明显改进;按约定未重复评论暂未上线的 4.14 链接,也未把 Task/Pipeline 合约随版本变化本身视为问题。

… consistency

Address the four Major review comments:
- failurePolicy/timeoutSeconds: the per-policy entry point is
  spec.webhookConfiguration (spec.failurePolicy is deprecated in 1.15);
  every policy asset now declares failurePolicy explicitly, the checklist
  reads declared intent plus generated webhook grouping, and ModuleInfo is
  scoped to platform-wide overrides only
- async cancellation monitoring: replaced the after-the-fact updaterequests
  query (deleted after retries, so always empty) with three verified data
  sources: kyverno_policy_results_total metrics, the background-controller
  ERR log signature, and a pre-started watch during fault drills; stopping
  the background-controller is restricted to dedicated clusters or
  maintenance windows
- upgrade regression step 3: two-phase flow (enumerate new override entry
  points, update the criterion, then verify blocking) instead of expecting
  a denylist to block fields it has never seen
- multi-cluster drift: baseline includes namespaced Policies and compares
  name + failureAction + match/exclude, aligned with section 7.3
@l-qing
l-qing deployed to translate August 13, 2026 05:48 — with GitHub Actions Active
- metrics count attempts, not incidents: drop the exact hits-vs-landings
  reconciliation claim, keep the error-increase alert, and point per-run
  attribution back at the target objects
- controller-down failures leave neither log lines nor new counters:
  liveness monitoring is the precondition signal, and the section 6.2
  troubleshooting order now starts there
- cross-cluster baseline and section 7.3 comparison now include
  spec.webhookConfiguration and the per-cluster forceFailurePolicyIgnore
  state; checklist item 6 and onboarding read namespaced Policies and the
  generated grouping, not the declaration alone
- soften 'after-the-fact query is invariably empty' to the retry-window
  semantics the evidence supports; align the checklist intro with the
  rewritten item 6
@l-qing
l-qing deployed to translate August 13, 2026 06:36 — with GitHub Actions Active
- failurePolicy tiering: state the 17 Fail / 8 Ignore split precisely and
  explain why the $4.2.2 cancellation trigger stays Fail (same admission
  gate as $4.2.1); annotate the asset
- $6.2.3 evidence script: print the decisive attribution fields first
  (parent spec.status / cancel-reason, per-TaskRun spec.status /
  statusMessage), wrap the block in a function so failed reads really stop
  collection, and fail explicitly on unparseable PolicyReport JSON
- mutate-existing RBAC scope: unified to $4.2.2 / $4.6.1 / $4.6.2 across
  $3.1, $4.0.1, $4.2.2 and the $4.6 preamble ($4.2.3 needs none)
- cancellation attribution: downgrade 'not yours = policy' to first
  suspect; note the API-server audit log as the way to confirm the writer
- $4.0.7 conversion: cover the demo-namespace pins outside match
  (mutate targets, companion Role/RoleBinding, $4.5.3 ConfigMap shape);
  fix the gate-param-mutate-to-cancel anchor to $4.2.3
- $3.8 step 9: add the same-profile compliant run as a healthy-side
  control; $3.4.1 bare-entry recipe: require the controller normal path
- en-zh parity: drop an en-only count sentence so the editions mirror
@l-qing
l-qing deployed to translate August 13, 2026 13:07 — with GitHub Actions Active
- Contracts 5/6 (DAG dominance, finally safety): guarantor corrected to
  template responsibility only; the shipped 4.1.4 audit checks gate
  identity, not runAfter/finally -- its resolved-definition snapshot is
  documented as the hook for building such an audit
- Cancellation markers: 'decisive' wording softened to attribution
  clues throughout; writer confirmation stays with the audit log
- PolicyException walkthrough asset: warn that governance metadata
  (ticket, validity window, owner) must be added for production issuance
- Audit-type acceptance row: add the healthy-side criterion (compliant
  input must yield pass/skip, no fail)
@l-qing
l-qing deployed to translate August 13, 2026 13:38 — with GitHub Actions Active
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants