From 4dd2db0cb153d18a66218c48a28d9b20802e7575 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 5 Aug 2026 06:05:30 +0800 Subject: [PATCH] docs: how to publish a library to mcpp-index, and the generated= path asymmetry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two gaps found by walking the chain end to end for libxpkg 0.0.48. Publishing a library was undocumented entirely — `mcpp publish` and `mcpp emit xpkg` appear in the CLI and nowhere in docs/. 09 covers releasing mcpp itself and 02 covers `mcpp pack`; neither says how a library becomes something `[dependencies]` can name. The new 10 writes down the chain and, more usefully, the four places it fails quietly: * gitcode answers HEAD with 401, so an asset verified with HEAD reads as broken and one verified by existence alone can still be the wrong bytes — GET it and cmp it against the GitHub tarball; * an index entry in one platform block resolves on one platform and fails on the others as "no such version", which reads like a consumer typo; * the index is an artifact, so merging to main publishes nothing until publish-artifact.yml runs, and clients hold a TTL on top of that; * a seeded local copy of the unreleased version is indistinguishable from a published one — it is exactly what is still there when the publish failed. The `generated=` row said "relative to the project root". True for the root package; a dependency's build.mcpp resolves the same path against MCPP_OUT_DIR. A library plays both roles, so no relative path is correct in both: writing to MCPP_OUT_DIR and emitting the bare name fails at the root with "declared generated source ... does not exist after the run". Documents the absolute-path form, and that a generated .cppm module interface works — the examples only showed .cpp. --- docs/07-build-mcpp.md | 32 ++++++- docs/10-publishing-a-library.md | 142 +++++++++++++++++++++++++++++ docs/README.md | 1 + docs/zh/07-build-mcpp.md | 27 +++++- docs/zh/10-publishing-a-library.md | 131 ++++++++++++++++++++++++++ docs/zh/README.md | 1 + 6 files changed, 332 insertions(+), 2 deletions(-) create mode 100644 docs/10-publishing-a-library.md create mode 100644 docs/zh/10-publishing-a-library.md diff --git a/docs/07-build-mcpp.md b/docs/07-build-mcpp.md index 6cfc9eac..78ae65c3 100644 --- a/docs/07-build-mcpp.md +++ b/docs/07-build-mcpp.md @@ -48,7 +48,7 @@ is ignored, so you can freely log diagnostics. | `mcpp:link-lib=` | link `-l` | | `mcpp:link-search=` | add a library search dir (`-L`; relative dirs resolve against the project root) | | `mcpp:cfg=` | define `-D` for both C and C++ | -| `mcpp:generated=` | add a generated source (relative to the project root) to the build | +| `mcpp:generated=` | add a generated source to the build. **A relative path resolves against the project root for the root package, but against `MCPP_OUT_DIR` for a dependency's build.mcpp** — emit an absolute path if the package is both (see below) | | `mcpp:source=` *(0.0.100+)* | select a **pre-existing** source file into the build (absolute, or relative to the package root). Same downstream effect as `generated=`; use it for files the program *chose* (payload/vendored tree) rather than wrote — e.g. a per-target source selection over a large tarball | | `mcpp:include-dir=` *(0.0.100+)* | add a **private** include directory (`-I`) for this package's own TUs (absolute, or relative to the package root; normalized). Replaces the `cxxflag=-I` + `cflag=-I` double emission | | `mcpp:include-dir-after=` *(0.0.100+)* | like `include-dir`, but searched **after** the system directories (`-idirafter`) — for payload trees that shadow system headers | @@ -165,6 +165,36 @@ cache, `MCPP_OUT_DIR`) live in the **consuming project's** across projects (and may be read-only), so it is never written to; relative `generated=` paths resolve against `MCPP_OUT_DIR`, not the package root. +### A library that is also built standalone: emit an absolute path + +Those two rules — project root for the root package, `MCPP_OUT_DIR` for a +dependency — mean a *relative* `generated=` cannot be right in both roles. A +library is built standalone by its own CI and consumed from the registry by +everyone else, so it plays both. + +Writing into `MCPP_OUT_DIR` and emitting the bare filename works as a +dependency and fails at the root with: + +``` +error: build.mcpp declared generated source 'foo.cppm' but it does not exist after the run +``` + +Write to `MCPP_OUT_DIR` (the package root may be read-only) and emit the +**absolute** path: + +```cpp +const auto out = std::filesystem::path(mcpp::out_dir()) / "foo.cppm"; +// ... write it ... +mcpp::generated(out.string().c_str()); +``` + +`mcpp::out_dir()` is always absolute, so this is correct in both roles and +needs no branch on which one you are in. + +A generated **module interface** is fine here: `.cppm` goes through the same +scan as any other source, so a generated file declaring `export module …` can +be imported by the package's own TUs. + ## Incremental: declared inputs (no needless re-runs) mcpp does **not** re-run `build.mcpp` on every build. It caches the program's diff --git a/docs/10-publishing-a-library.md b/docs/10-publishing-a-library.md new file mode 100644 index 00000000..06b72483 --- /dev/null +++ b/docs/10-publishing-a-library.md @@ -0,0 +1,142 @@ +# 10 - Publishing a Library to mcpp-index + +**English** | [简体中文](zh/10-publishing-a-library.md) + +How a library becomes something `[dependencies]` can name. This is the +*library author's* chain; [09 - Releasing mcpp](09-release.md) is about +releasing mcpp itself, and [02 - Packaging & Release](02-pack-and-release.md) +is about `mcpp pack` bundling an application. + +## The chain, in the only order that works + +``` +your repo merge → git tag → GitHub auto-generates the tag tarball + ↓ +gitcode mirror a byte-identical copy, for the CN region + ↓ +mcpplibs/mcpp-index pkgs//.lua — GLOBAL + CN URLs + sha256 + ↓ publish-artifact.yml pushes a content-hash artifact + ↓ +consumers bump the version in their mcpp.toml +``` + +Each arrow is a gate. Skipping one does not fail loudly — it fails as +"dependency not found" or a 404 in someone else's build, hours later. + +## 1. Tag the release + +The version in `mcpp.toml` and the tag must agree. GitHub generates +`archive/refs/tags/.tar.gz` automatically; that tarball **is** the +artifact — nothing needs uploading. + +```bash +git tag 0.0.48 && git push origin 0.0.48 +curl -fsSL -o pkg-0.0.48.tar.gz \ + https://github.com///archive/refs/tags/0.0.48.tar.gz +sha256sum pkg-0.0.48.tar.gz # ← the digest the index will carry +``` + +The tarball extracts to `-/`, and mcpp looks for `mcpp.toml` inside +that wrapper directory. A repo that ships its own `mcpp.toml` needs no `mcpp` +field in the index entry. + +## 2. Mirror to gitcode + +The CN entry must be a **byte-identical copy** of the GitHub tarball, only +renamed. Anything else and the two regions disagree about what a pinned +`sha256` means. + +```bash +gtc release publish mcpp-res/ --tag 0.0.48 --asset pkg-0.0.48.tar.gz +``` + +Then verify it, because the upload reporting success is not the same as the +asset being fetchable: + +```bash +# GET, never HEAD — gitcode answers HEAD with 401 and GET with 302 → CDN 200 +curl -fsSL -o cn.tar.gz \ + https://gitcode.com/mcpp-res//releases/download/0.0.48/pkg-0.0.48.tar.gz +cmp cn.tar.gz pkg-0.0.48.tar.gz # must be identical, not merely present +``` + +## 3. Add the index entry + +In `mcpplibs/mcpp-index`, `pkgs//.lua`: + +```lua +["0.0.48"] = { + url = { + GLOBAL = "https://github.com///archive/refs/tags/0.0.48.tar.gz", + CN = "https://gitcode.com/mcpp-res//releases/download/0.0.48/pkg-0.0.48.tar.gz", + }, + sha256 = "", +}, +``` + +**In all three platform blocks** — `linux`, `macosx`, `windows`. A source +tarball is the same bytes on every platform, and an entry present in only one +of them fails on the others as "no such version", which reads like a typo in +the consumer's manifest. + +## 4. Wait for the artifact + +**The index is an artifact, not a git clone.** Merging to `main` is not +enough: `publish-artifact.yml` has to run and push a content-hash artifact, +and clients hold a refresh TTL on top of that. Editing a cached `pkgs/**` by +hand does nothing. + +```bash +gh run list --repo mcpplibs/mcpp-index --workflow publish-artifact.yml --limit 1 +rm -rf ~/.mcpp/registry/data/ # force a client refresh +``` + +## 5. Verify from a cold resolve, then bump consumers + +The point of this step is that a local checkout of the library will mask every +mistake above. Resolve it the way a stranger would: + +```bash +rm -rf ~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +mcpp build # must download and compile 0.0.48 +``` + +Only then bump `[dependencies]` in the consumers. + +> Do **not** try to force a refresh with +> `find ~/.mcpp/registry -mindepth 1 -maxdepth 1 ! -name data -exec rm -rf {} +` +> alone. `data/xpkgs` sits at depth 2 and is not named `data`, so a careless +> second pass deletes the whole payload store (~800 MB of toolchains). +> Recovery is `mcpp self doctor`, which re-provisions, then `mcpp update`. + +## Testing against an unreleased version + +While the chain above is still in flight, seed the registry by hand so +consumers can compile against the library before it is published: + +```bash +REG=~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +mkdir -p "$REG" +git -C /path/to/library archive --format=tar --prefix=-0.0.48/ HEAD \ + | tar -x -C "$REG" +touch "$REG/.mcpp_ok" # the marker that says "resolved" +cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # add a 0.0.48 entry to it +``` + +mcpp's build sandbox is network-isolated, so `file://` and +`http://127.0.0.1` index URLs cannot be fetched — seeding the cache is the way. + +**Remove the seeded copy before believing the real thing works.** A seeded +0.0.48 and a published 0.0.48 are indistinguishable to the build, and the +seeded one is the copy that will still be there when the publish silently +failed. + +## Checklist + +- [ ] `mcpp.toml` version == git tag +- [ ] tag pushed; tarball downloads and its sha256 recorded +- [ ] gitcode asset verified with **GET**, byte-identical to GitHub's +- [ ] index entry in **all three** platform blocks +- [ ] `publish-artifact.yml` succeeded +- [ ] cold resolve (seeded copy deleted) downloads and compiles it +- [ ] consumers bumped diff --git a/docs/README.md b/docs/README.md index f3d8ee53..4d8637f0 100644 --- a/docs/README.md +++ b/docs/README.md @@ -12,6 +12,7 @@ - [07 - build.mcpp Build Program](07-build-mcpp.md) - [08 - Toolchain Internals](08-toolchain-internals.md) - [09 - Releasing mcpp](09-release.md) +- [10 - Publishing a Library to mcpp-index](10-publishing-a-library.md) ## Specifications diff --git a/docs/zh/07-build-mcpp.md b/docs/zh/07-build-mcpp.md index 22391de8..f62e53fb 100644 --- a/docs/zh/07-build-mcpp.md +++ b/docs/zh/07-build-mcpp.md @@ -45,7 +45,7 @@ mcpp build # 编译 + 运行 build.mcpp,然后构建工程 | `mcpp:link-lib=` | 链接 `-l` | | `mcpp:link-search=` | 增加库搜索目录(`-L`;相对路径按工程根目录解析) | | `mcpp:cfg=` | 为 C 与 C++ 同时定义 `-D` | -| `mcpp:generated=` | 把生成的源码(相对工程根目录)加入构建 | +| `mcpp:generated=` | 把生成的源码加入构建。**相对路径在根工程按工程根解析,在依赖的 build.mcpp 里按 `MCPP_OUT_DIR` 解析** —— 两种角色都可能出现的包应发绝对路径(见下文) | | `mcpp:source=` *(0.0.100+)* | 把一份**既有**源文件选入构建(绝对路径,或相对包根)。下游效果与 `generated=` 相同;语义区别在于文件是程序*选中*的(tarball payload / vendored 源树)而非程序写出的——例如对大型源码包做 per-target 源选择 | | `mcpp:include-dir=` *(0.0.100+)* | 为本包自身 TU 增加一个**私有** include 目录(`-I`;绝对路径或相对包根,自动规范化)。取代过去 `cxxflag=-I` + `cflag=-I` 的双重裸发 | | `mcpp:include-dir-after=` *(0.0.100+)* | 同 `include-dir`,但排在系统目录**之后**搜索(`-idirafter`)——用于会遮蔽系统头的 payload 源树 | @@ -152,6 +152,31 @@ mcpp 会把它自己构建时用的**同一份** std 模块暂存过来,缓存 `target/.build-mcpp/deps/@/` 下——registry 包根跨工程共享(且可能只读), 绝不写入;相对 `generated=` 路径按 `MCPP_OUT_DIR` 解析,而非包根。 +### 既独立构建又被当依赖的库:发绝对路径 + +上面这两条规则——根工程按工程根、依赖按 `MCPP_OUT_DIR`——意味着**相对** +`generated=` 不可能两种角色都对。而一个库正好两种角色都有:自己的 CI 独立构建它, +别人从 registry 当依赖用它。 + +写进 `MCPP_OUT_DIR` 再发裸文件名,在依赖角色下能用,在根工程下则失败: + +``` +error: build.mcpp declared generated source 'foo.cppm' but it does not exist after the run +``` + +正确做法是写进 `MCPP_OUT_DIR`(包根可能只读),并发**绝对**路径: + +```cpp +const auto out = std::filesystem::path(mcpp::out_dir()) / "foo.cppm"; +// ... 写文件 ... +mcpp::generated(out.string().c_str()); +``` + +`mcpp::out_dir()` 恒为绝对路径,因此两种角色下都正确,不需要判断自己处在哪一种。 + +生成**模块接口**是可以的:`.cppm` 走与其他源文件相同的扫描,所以一个生成出来的、 +声明 `export module …` 的文件可以被该包自己的 TU import。 + ## 增量:声明输入(避免无谓重跑) mcpp **不会**每次构建都重跑 `build.mcpp`。它会缓存程序产出的指令,只有当它依赖的东西 diff --git a/docs/zh/10-publishing-a-library.md b/docs/zh/10-publishing-a-library.md new file mode 100644 index 00000000..11dcf59e --- /dev/null +++ b/docs/zh/10-publishing-a-library.md @@ -0,0 +1,131 @@ +# 10 - 发布一个库到 mcpp-index + +[English](../10-publishing-a-library.md) | **简体中文** + +一个库如何变成 `[dependencies]` 可以写出来的东西。这是**库作者**的链路; +[09 - 发布 mcpp](09-release.md) 讲的是发布 mcpp 自身, +[02 - 发布打包](02-pack-and-release.md) 讲的是 `mcpp pack` 打包应用。 + +## 这条链,只有一种顺序成立 + +``` +你的仓库 merge → git tag → GitHub 自动生成 tag tarball + ↓ +gitcode 镜像 逐字节相同的一份拷贝,供 CN 区 + ↓ +mcpplibs/mcpp-index pkgs//.lua —— GLOBAL + CN 双 URL + sha256 + ↓ publish-artifact.yml 推出内容哈希 artifact + ↓ +消费方 在自己的 mcpp.toml 里升版本 +``` + +每一支箭头都是一道关。漏掉任何一道都不会当场报错 —— 它会在几小时后, +以别人构建里的 "dependency not found" 或一个 404 的形式出现。 + +## 1. 打 tag + +`mcpp.toml` 里的版本必须与 tag 一致。GitHub 会自动生成 +`archive/refs/tags/.tar.gz`,**那个 tarball 就是产物**,不需要另行上传。 + +```bash +git tag 0.0.48 && git push origin 0.0.48 +curl -fsSL -o pkg-0.0.48.tar.gz \ + https://github.com///archive/refs/tags/0.0.48.tar.gz +sha256sum pkg-0.0.48.tar.gz # ← 索引里要写的摘要 +``` + +tarball 解开是 `-/`,mcpp 会在这层包装目录里找 `mcpp.toml`。 +仓库自带 `mcpp.toml` 时,索引条目不需要 `mcpp` 字段。 + +## 2. 镜像到 gitcode + +CN 条目必须是 GitHub tarball 的**逐字节拷贝**,只改文件名。否则两个区域对同一个 +`sha256` 的理解就不一致了。 + +```bash +gtc release publish mcpp-res/ --tag 0.0.48 --asset pkg-0.0.48.tar.gz +``` + +然后**验证**它 —— 上传报成功和资源真的能取到,不是一回事: + +```bash +# 用 GET,绝不用 HEAD —— gitcode 对 HEAD 返回 401,对 GET 返回 302 → CDN 200 +curl -fsSL -o cn.tar.gz \ + https://gitcode.com/mcpp-res//releases/download/0.0.48/pkg-0.0.48.tar.gz +cmp cn.tar.gz pkg-0.0.48.tar.gz # 必须一致,而不只是"存在" +``` + +## 3. 加索引条目 + +在 `mcpplibs/mcpp-index` 的 `pkgs/<首字母>/.lua`: + +```lua +["0.0.48"] = { + url = { + GLOBAL = "https://github.com///archive/refs/tags/0.0.48.tar.gz", + CN = "https://gitcode.com/mcpp-res//releases/download/0.0.48/pkg-0.0.48.tar.gz", + }, + sha256 = "<第 1 步得到的摘要>", +}, +``` + +**三个平台块都要写** —— `linux`、`macosx`、`windows`。源码 tarball 在每个平台上是 +同样的字节;只写了其中一个,在另外两个平台上会以 "no such version" 失败, +而那读起来像是消费方 manifest 里打错了字。 + +## 4. 等 artifact + +**索引是 artifact,不是 git clone。** 合进 `main` 还不够:必须等 +`publish-artifact.yml` 跑完并推出内容哈希 artifact,客户端之上还有一层刷新 TTL。 +手改缓存里的 `pkgs/**` 不起任何作用。 + +```bash +gh run list --repo mcpplibs/mcpp-index --workflow publish-artifact.yml --limit 1 +rm -rf ~/.mcpp/registry/data/ # 强制客户端刷新 +``` + +## 5. 冷解析验证,然后再升消费方 + +这一步的意义在于:本地那份库的 checkout 会掩盖上面每一个错误。要像一个陌生人那样解析它: + +```bash +rm -rf ~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +mcpp build # 必须真的下载并编译 0.0.48 +``` + +通过之后,才去升消费方的 `[dependencies]`。 + +> **不要**只用 +> `find ~/.mcpp/registry -mindepth 1 -maxdepth 1 ! -name data -exec rm -rf {} +` +> 去强制刷新。`data/xpkgs` 在第 2 层且名字不是 `data`,再来一遍粗心的清理就会把整个 +> payload 仓(约 800 MB 工具链)删掉。恢复办法是 `mcpp self doctor` 重新 provision, +> 再 `mcpp update`。 + +## 对着尚未发布的版本做测试 + +在上面这条链还没走完时,可以手工播种 registry,让消费方提前编译: + +```bash +REG=~/.mcpp/registry/data/xpkgs/-x-/0.0.48 +mkdir -p "$REG" +git -C /path/to/library archive --format=tar --prefix=-0.0.48/ HEAD \ + | tar -x -C "$REG" +touch "$REG/.mcpp_ok" # 表示"已解析"的标记 +cp ../0.0.47/.xpkg.lua "$REG/.xpkg.lua" # 往里加一条 0.0.48 条目 +``` + +mcpp 的构建沙箱是网络隔离的,`file://` 和 `http://127.0.0.1` 形式的索引 URL 取不到, +播种缓存才是可行的办法。 + +**在相信"真的能用"之前,先把播种的那份删掉。** 播种的 0.0.48 和已发布的 0.0.48 +对构建来说毫无区别 —— 而当发布其实失败了的时候,留在那里的正是播种的那一份。 + +## 检查清单 + +- [ ] `mcpp.toml` 版本 == git tag +- [ ] tag 已推;tarball 可下载,sha256 已记录 +- [ ] gitcode 资源用 **GET** 验证过,且与 GitHub 那份逐字节一致 +- [ ] 索引条目写进了**三个**平台块 +- [ ] `publish-artifact.yml` 成功 +- [ ] 冷解析(删掉播种拷贝后)能下载并编译 +- [ ] 消费方已升版本 diff --git a/docs/zh/README.md b/docs/zh/README.md index 27518dda..511a32c9 100644 --- a/docs/zh/README.md +++ b/docs/zh/README.md @@ -12,3 +12,4 @@ - [07 - build.mcpp 构建程序](07-build-mcpp.md) - [08 - 工具链机制内幕](08-toolchain-internals.md) - [09 - 发布 mcpp](09-release.md) +- [10 - 发布一个库到 mcpp-index](10-publishing-a-library.md)