Skip to content

feat: Helm charts for the operator and the platform - #37

Open
TheMeinerLP wants to merge 17 commits into
mainfrom
feat/helm-charts
Open

feat: Helm charts for the operator and the platform#37
TheMeinerLP wants to merge 17 commits into
mainfrom
feat/helm-charts

Conversation

@TheMeinerLP

Copy link
Copy Markdown
Contributor

Implements the approved design in docs/superpowers/specs/2026-08-13-helm-charts-design.md. Apus had no deployment description at all; the Phase 8 plan's Kustomize base is replaced by two Helm charts.

What lands

  • deploy/charts/apus-operator — the six CRDs, the controller, cluster-wide RBAC. The minimum Apus needs; usable on its own by anyone driving it through kubectl.
  • deploy/charts/apus-platform — REST API and dashboard behind one ingress.
  • Both versioned by release-please in the root track, so apus-operator-0.3.0 necessarily deploys apus/operator:0.3.0image.tag is empty and falls back to .Chart.AppVersion.
  • Published as OCI artifacts to Harbor, matching how the cluster repo already consumes the kube-prometheus-stack via OCIRepository.
  • A PR-build job that lints, renders and schema-checks both charts.

runner, ingest and hosting get no chart: the operator creates them from custom resources. Helm only passes their image references through.

CRDs ship as templates, not in crds/

Helm installs its crds/ directory once and never touches it again, so helm upgrade would leave an old schema in place while a new operator reads fields it does not know. They are templates with helm.sh/resource-policy: keep instead — updated on upgrade, kept on uninstall so that removing the chart does not delete every Tenant and BlueMapMap in the cluster.

What the final review caught

It stood up a disposable k3s, installed both charts, upgraded to a simulated 0.3.0, uninstalled, and queried the RBAC with SubjectAccessReview. Three defects that no per-task review could have seen:

  • The operator could not create Secrets. The ClusterRole granted get,list,watch with a comment claiming it "only ever reads them" — but TenantReconciler creates the apus-push-token Secret in every tenant namespace. Every Tenant reconcile would have 403'd, breaking the very first step NOTES.txt tells a user to take.
  • No cert-manager.io rule, so BlueMapHosting with TLS would fail on any cluster that actually has cert-manager.
  • The PR job could not pass in CI. kubectl apply --dry-run=client is not offline — it downloads the OpenAPI document from an API server. It passed locally only because this machine has a kubeconfig. Replaced with pinned kubeconform.

Plus: auth.jwksUri is now enforced like auth.issuer (without it the API rejects every token at runtime instead of failing at install), and the chart push no longer uses helm registry login — see below.

A lead on the Harbor problem

The image pushes currently fail with empty challenge header. The review found the likely cause: the org's docker-publish.yml uses regctl ... --skip-check, commented "skips its anonymous connectivity ping, which a private registry answers with 401". helm registry login performs exactly that ping. The chart publish therefore writes the auth entry directly and passes --registry-config to helm push, avoiding it. Verified against a local registry:2 with htpasswd — not against the real Harbor, which is still unreachable this way.

Two gaps recorded, not fixed

Both are outside the charts and are now numbered open points in the design spec §11:

  • The dashboard cannot be configured. ui/nuxt.config.ts defaults oidcIssuer/oidcClientId to '', and the Dockerfile runs pnpm generate with no build args while copying only .output/public into nginx — so NUXT_PUBLIC_* cannot apply at runtime without a Nitro server. No installation can currently log in to the UI.
  • Operator-created workloads have no image pull secret, though their images default to a private Harbor project.

🤖 Generated with Claude Code

Two charts in this repository, versioned with the code and published as OCI
artifacts to Harbor: apus-operator (CRDs plus controller, the minimum Apus
needs) and apus-platform (API and dashboard). runner, ingest and hosting get
no chart of their own -- the operator creates them from custom resources.

CRDs ship as templates with helm.sh/resource-policy: keep rather than in
Helm's crds/ directory, which is never updated on upgrade.
…ager certificates

The ClusterRole granted secrets get/list/watch with a comment claiming the operator
only ever reads them. TenantReconciler creates the apus-push-token Secret in every
tenant namespace on first reconcile, so the very first Tenant -- the first thing
NOTES.txt tells a user to create -- failed with a 403.

The same role had no cert-manager.io rule at all, while BlueMapHostingReconciler does
createOr(update) on a Certificate whenever a BlueMapHosting has TLS enabled. On a
cluster with cert-manager installed, every TLS hosting failed the same way.

Both grants are as narrow as RBAC allows: the push-token secret has a fixed name but
an unbounded set of namespaces, which resourceNames cannot express, and neither
resource is ever updated, patched or deleted beyond what createOr(update) needs.
Also corrects the namespace rule's comment -- TenantReconciler creates no
NetworkPolicy today, the rule is ahead of the code.
`kubectl apply --dry-run=client` is not an offline check: since kubectl 1.26 it
downloads the OpenAPI document from the API server before validating, so on a runner
with no cluster it resolves to localhost:8080 and dies with "failed to download
openapi ... connection refused". The step only ever passed because it was run on a
machine with a live kubeconfig. kubeconform validates against published JSON schemas
and never contacts an API server; the step is named for what it now does.

auth.jwksUri is as mandatory as auth.issuer -- application.yml wires both from the
environment with no default -- but nothing enforced it, so an install without it
produced an API that rejects every token at runtime. It is now required by the same
schema and guarded by its own CI assertion, run with the issuer supplied so the
assertion cannot pass on the issuer error alone.
`helm registry login` opens with an anonymous connectivity ping, which the Harbor
behind this registry answers with a 401 that carries no WWW-Authenticate header --
the `empty challenge header` failure the image push hit until docker-publish.yml
switched to `regctl registry login --skip-check`. Helm has no --skip-check, so the
same idea is applied one level down: write the credential file the login would have
written and pass it to `helm push` with --registry-config, which authenticates on the
push request itself. Keeping helm as the pushing client means the artifact stays
exactly what `helm pull` expects, instead of hand-assembling chart media types with
regctl.

Verified against a local registry with basic auth: the push succeeds with the
hand-written config and no login, and fails with "basic credential not found"
without it. It could not be verified against the real Harbor.

Also pins azure/setup-helm to the same v4.2.2 as build-pr.yml, so the chart that
ships is packaged by the Helm the PR job linted it with.
sync-crds.sh is a developer script that copies generated CRDs into the chart; it has
no purpose inside the published package and is now ignored (confirmed with tar -tzf
after helm package).

The operator's NOTES.txt claimed a Tenant provisions a network policy (TenantReconciler
creates none) and that the /metrics endpoint exists and Prometheus will scrape it (the
operator has no HTTP server at all until Phase 8 Task 4 -- a scrape gets connection
refused, not empty data).

Both READMEs now document that reinstalling under a different release name fails on
the CRDs, which carry meta.helm.sh/release-name from the first install and survive
uninstall through helm.sh/resource-policy: keep, plus the two ways out. The platform
README says plainly that its own resources are unaffected rather than repeating the
warning as if it applied there.
…ask 2

Phase 9 Task 2 pointed at deploy/base/api-rbac.yaml, a Kustomize file this branch made
sure will never exist. The API RBAC now lives only in the apus-platform chart, whose
own comment hands the secret narrowing off to that task.

Design spec §11 gains two open points that are real but out of the charts' reach: the
dashboard cannot be configured at all (empty OIDC values are frozen into the image,
NUXT_PUBLIC_* needs a Nitro server the nginx image does not contain, so no installation
can log in), and nothing gives the operator-created Jobs and Deployments an image pull
secret while their images default to a private Harbor project. Both fixes belong in the
UI build and the operator code respectively.

§11.1 and §11.4 are updated to what the branch actually built, including that the chart
push has not been exercised against the real registry.
@github-actions

Copy link
Copy Markdown
Contributor

Test results

496 tests   496 ✅  37s ⏱️
 68 suites    0 💤
 68 files      0 ❌

Results for commit 7076a20.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant