docs(rfc): propose Bazel build system

[SDAChess]

Supporting document for #2491

[copy-pr-bot]

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

[krishicks]

It might be good to explain what complexity is being referred to here; a lot of historical complexity in Bazel has been removed e.g. through tooling like Bzlmod. We should make sure our assessment of Bazel is grounded in the experience today, not an experience someone may have had years ago.

[krishicks]

Similarly, explain what the actual cost of adopting Bazel is in concrete terms as it relates to complexity or otherwise.

[varshaprasad96]

+1 to @krishicks. The historical concerns with Bazel were non-hermetic behavior due to actions downloading pre-compiled binaries, manual dependency mirroring, and the issues of maintaining reproducible builds However, modern Bazel (9+) with Bzlmod addresses the core hermeticity issues through proper module resolution and dependency pinning.

From productization and downstream perspective of RH, we are fine with Bazel, provided we are considering latest Bazel + bazelmod. As of now, we have setup pipelines with cargo workflows from main, and can transition to Bazel if there is sufficient overlap in time

[rhuss]

The RFC is well-written and honest about the risks, which I appreciate. I want to engage seriously with the arguments because this decision has long-term consequences for contributor experience and downstream consumption.

The dual-system problem

The RFC acknowledges dual-system drift under "Risks" but doesn't resolve it. It proposes "bounded coexistence" with exit criteria. Kubernetes proposed the same thing. That coexistence lasted years. The community removed Bazel in kubernetes/kubernetes#99561 after 73 comments of discussion (see also KEP-2420 and the community discussion), and the reasons were operational, not theoretical: every dependency update required changes to both systems, maintaining two systems cost more than either one was worth, and divergent builds created qualification gaps that made CVE fixes on older branches "significantly more complicated."

The RFC frames #2414 as "evidence for Phase 1, not approval of the full migration." But #2414 is now merged, and #2595 is already adding more Bazel targets. BUILD files in the repo create gravitational pull. Contributors start depending on them. Tooling adapts. Removing them gets harder every week. This is the pattern K8s experienced.

Tim Hockin (K8s co-founder): "The goal with Bazel was to get it to the place where 99% of developers never interact with it. I think we never got there."

Rust developer experience

The RFC says Cargo "remains the right package manager and native developer interface for individual Rust crates." In practice, adopting Bazel means one of two outcomes:

1. Maintain Cargo.toml AND BUILD.bazel files (the exact dual-system problem K8s warned about)
2. Make Bazel authoritative and lose native cargo clippy, cargo fmt, cargo expand, and rust-analyzer integration

rust-analyzer integration with Bazel has known issues at scale. DFINITY found that rust-analyzer "worked perfectly in the prototype but choked on their code base," and they still keep Cargo files around for IDE users. The rules_rust performance issue #962 confirms the rust_analyzer rule degrades with many targets and deep dependency hierarchies. Leftover build artifacts can break rust-project.json generation entirely, and auto-discovery via discoverConfig is still being integrated as of 2026.

Teams that adopt Bazel for Rust consistently keep Cargo files around for IDE users, recreating the dual-system problem. The RFC's "Developer and agent experience" section resolves this tension by arguing that "AI coding agents reduce repetitive work." That's betting the build system on an assumption about AI tooling maturity. What happens when the agent gets the Starlark wrong and a human has to debug it?

OpenShell uses cargo clippy, cargo fmt, and cargo expand throughout the codebase. These have no Bazel equivalents. Losing them, or maintaining them through a parallel Cargo path, is a cost the RFC should quantify.

Expert pool

K8s found that "only a handful of people could debug Bazel issues." When those people were unavailable, everyone else was blocked. Their #bazel Slack channel was full of questions about things that should have been straightforward.

OpenShell needs external contributors. The RFC's mitigation for contributor barriers ("documented common paths, small BUILD files, limited custom Starlark") is aspirational. K8s tried that approach and concluded it wasn't enough.

The RFC also argues AI agents change this equation. Maybe. But building a project's build system on the assumption that AI will compensate for human accessibility gaps is a bet I'd want to see validated before committing to, not after.

xtask+lima isn't evaluated

PR #2254 introduces xtask for orchestrating tests in lima VMs. The RFC's "Expand Cargo orchestration" alternative describes this approach briefly but frames it as "building its own heterogeneous task graph" and moves on. It doesn't engage with what #2254 actually delivers or compare it against the requirements from #2204.

xtask keeps build logic in Rust (the language this project uses), preserves the full Cargo toolchain, and addresses the CI-local parity problem that #2204 describes. lima provides real OS environments for packaging validation. Together they cover the core of #2204 without introducing Starlark, a new build model, or a new expert-pool dependency.

The RFC should evaluate xtask+lima against the same criteria it uses for Bazel, not dismiss it as what you do "if Bazel's complexity is rejected."

Downstream rebuild

OpenShell will be consumed by organizations running their own hermetic build pipelines that enforce network isolation during builds. Bazel 8 does add vendor mode and --nofetch to support offline builds, which is real progress. But there are two practical gaps:

First, enterprise build pipelines don't adopt the latest Bazel version quickly. Tinder's Bazel migration took 18 months to reach their first production build. Organizations with strict compliance requirements typically lag 6-12 months behind upstream releases. The vendor mode improvements in Bazel 8 may not be available in the Bazel versions that downstream rebuilders actually run.

Second, even with Bazel 8's offline support, a 2025 IEEE study analyzed 70 Bazel-using open-source projects and found that none had a completely hermetic build process. 71.9% of non-hermetic dependencies were Linux utility toolchains, and 38.1% were introduced by default configurations of Bazel build rules. This isn't a Bazel design flaw, it's a practical gap between what Bazel offers and what projects achieve.

Cargo's lockfile-based model works with network-isolated build systems out of the box, no vendor mode needed, no version dependency. The RFC's "Practical hermeticity" section focuses on OpenShell's own cache trust but doesn't address how downstream rebuilders interact with the project.

Questions

1. What's the maximum coexistence duration? The RFC says "bounded" but doesn't define the bound. K8s coexisted for years.
2. If rust-analyzer integration remains experimental at the end of Phase 1, does that block Phase 2? IDE support isn't listed as a gate.
3. How many contributors today can debug Bazel build failures? How many need to before the Mise path is removed?
4. Has xtask+lima (PR #2254) been evaluated against the same acceptance criteria? If not, can it be evaluated before this RFC is accepted?
5. How do downstream consumers rebuild OpenShell in network-isolated environments? The RFC should address this.