cratebank

An unofficial and ongoing census of Rust compilation times.

We all want Rust to compile faster. So why are we throwing out the data that might hold the key to making that happen? cratebank collects information from developers and organizations about Rust compilation times, processes the data so that it is easy to work with, then publishes datasets that anyone can use for free. People volunteer this information with the hope that it can be used to improve Rust generally and Rust compile times specifically.

The site is https://cratebank.io/; the data is at https://data.cratebank.io/.

Getting started

It's as easy as installing two crates and then running the right command.

cargo install cargo-cratebank
cargo install samply          # the profiler that measures compiler phases

cargo cratebank build         # builds, measures, and sends compile time info

Pre-launch. cargo-cratebank is not on crates.io yet. Until it is, install it straight from the repository: cargo install --git https://github.com/PowderworksCode/cratebank cargo-cratebank

On many Linux distributions samply needs permission to read perf events, once per boot:

sudo sysctl kernel.perf_event_paranoid=1

Arguments after build pass through to Cargo:

cargo cratebank build --release --all-features

To see exactly what would be sent from your repository, and send nothing:

cargo cratebank --dry-run build

cargo cratebank status checks that samply is installed, reports whether this client is current, and prints the active endpoint, machine id, and organization id. cargo cratebank serve runs a local reference collector, so the whole path can be exercised with no infrastructure; it is behind the off-by-default serve feature, so build it with cargo install cargo-cratebank --features serve.

The client checks whether a newer version has been published and says so once the build is done. It asks on its own thread, so it costs no time; it never blocks or fails an upload; it is not asked at all when you point cratebank at your own collector; and CRATEBANK_NO_UPGRADE_CHECK turns it off.

Sending every build

For automatic sending on macOS and Linux, add this to your ~/.zshrc or ~/.bashrc:

cargo() {
    if [[ "$1" == "build" ]]; then
        shift
        command cargo cratebank build "$@"
    else
        command cargo "$@"
    fi
}

On Windows, run notepad $PROFILE and add this:

function cargo {
    if ($args.Count -gt 0 -and $args[0] -eq "build") {
        $rest = @()
        if ($args.Count -gt 1) {
            $rest = $args[1..($args.Count - 1)]
        }

        & cargo.exe cratebank build @rest
    }
    else {
        & cargo.exe @args
    }
}

Every cargo build then runs under the sampler and contributes an observation. If measurement or the upload fails — an offline laptop, a collector having a bad day — cratebank says so and your build still succeeds; once your code has compiled, nothing here changes the exit code. Pass --strict to turn those warnings back into errors. Windows sampling has not yet been verified end to end, so treat the PowerShell function as experimental.

From CI

Replace your toolchain-setup and build steps with one action:

- uses: PowderworksCode/cratebank/action@v1
  with:
    args: --release --all-features

The action installs the stable toolchain, samply, and this client, grants perf permissions on Linux runners, builds under both, and uploads the filtered payload. Inputs are args, toolchain, working-directory, endpoint, machine-id (default ci), org-id, and strict. strict passes --strict to the client, so a CI job that set out to contribute fails when measurement fails instead of quietly warning. On non-Linux hosted runners, where sampling cannot run, the action falls back to a plain cargo build unless strict is set.

What gets sent

• Public crates only. Anything that is not from crates.io or a public git remote is dropped entirely — not the name, not a hash, not a timing. Anonymous counts report how many units were withheld and how many Cargo artifacts were fresh or rebuilt, so cache state is usable without exposing private identity.

• Even top level open source crates are private by default. Publishing them takes an explicit opt-in:

  [package.metadata.cratebank] # or [workspace.metadata.cratebank]
  public = true
  repository = "https://github.com/you/project"

• Build behavior. Unit and phase timing, dependency edges, emitted output kinds and sizes, end-to-end duration, sampled peak RSS, cache behavior, and sanitized build configuration.

• The machine. CPU model, core count, memory to the nearest gigabyte, kernel and OS version, architecture, virtualization hint, compiler and Cargo versions, and machine load during the build — the kind of detail a hardware review prints. Never a hostname, username, or network identity.

• A machine id. Generated locally on first use and stored in $CARGO_HOME/cratebank/machine-id. It is what makes within-machine comparison possible, and equally what makes a build history linkable; set CRATEBANK_MACHINE_ID to a role such as ci or laptop, or to none to send no id at all. Organization attribution is opt-in and absent by default.

• No paths, ever. Working directory, target directory and manifest path are never collected; compiler flags are kept but their path values are not.

Nothing wraps rustc, so there is no conflict with sccache or any other RUSTC_WRAPPER. Data is sent only when you invoke cargo cratebank build, and only when the sampled build, the Cargo timing report, and the samply profile all parse — otherwise cratebank warns and sends nothing.

On success the client prints the URL of the observation it uploaded, so you can read back exactly what you contributed. The raw uploads are published too, not only the processed tables: every session blob is world-readable under https://data.cratebank.io/sessions/, and sessions.parquet carries the key of the blob each row came from.

Configuring it

Endpoint, organization, and machine identity can live with the project:

[workspace.metadata.cratebank] # or [package.metadata.cratebank]
endpoint = "https://collector.example/v1/sessions"
org_id = "acme"        # optional; omitted by default
machine_id = "ci"      # or "laptop", "workstation", etc.

An explicit command-line flag wins over its environment variable, which wins over package metadata, then workspace metadata. The relevant environment variables are CRATEBANK_ENDPOINT, CRATEBANK_ORG_ID, and CRATEBANK_MACHINE_ID.

Using the data

Everything is public parquet on R2 — no account or API key needed. Use the files however you like; these scripts just make it easy with DuckDB.

curl -fsSL https://raw.githubusercontent.com/PowderworksCode/cratebank/main/docs/install.sql | duckdb cratebank.duckdb
duckdb cratebank.duckdb

Once the views are installed, queries are ordinary SQL:

SELECT package, phase, sum(samples) AS samples
FROM phases
WHERE thread = 'serial'
GROUP BY 1, 2
ORDER BY samples DESC;

The tables, rebuilt daily from every session collected so far:

• sessions.parquet — one row per build
• units.parquet — one row per compilation unit
• phases.parquet — sampled compiler phases per unit
• timeline.parquet — build concurrency and CPU over time
• unit_flags.parquet — the settings each unit was built with
• artifacts.parquet — output kinds and byte sizes per compilation unit
• edges.parquet — dependency edges for critical-path analysis
• compiler_units.parquet — rustc wall time and sampled peak RSS
• build_config.parquet — sanitized session-level build configuration

schema/v1/tables.json describes every column and carries the warnings that matter — chiefly that sampled phases are CPU while the section boundaries in units are wall clock, and the two are not interchangeable.

Download the Python starter notebook for a guided analysis of sessions, package wall time, compiler phases, build settings, and one build timeline.

How it works

One cargo build --timings runs under samply, and both outputs are parsed from that same build. Cargo supplies wall-clock unit measurements, dependency edges, and concurrency; samply supplies the CPU-weighted compiler-phase breakdown, attributed per rustc process through the -C metadata= hash on its command line. Peak RSS is sampled externally, from outside the build, which is why no compiler wrapper is needed.

docs/ carries the details: collection.md for what one contribution is, capture-manifest.md for every field and its source, schema.md for the public tables, ingest.md for the service, and handoff.md for operations. infra/README.md covers the Cloudflare stack and deployment.

Licensing

The source code and repository documentation are available under the MIT License. The public dataset is available under the Creative Commons Attribution 4.0 International License. Credit "cratebank contributors," link to https://cratebank.io/ and to the license, and say whether you changed the data.

By uploading an observation, you confirm that you may contribute it under CC BY 4.0. Only submit observations you have the right to contribute.

Contact

Zack got tired of waiting on Rust to compile, so he made cratebank as part of his open source work on The Powderworks Agentic Code Consortium. The best way to get in touch about it is the Powderworks Zulip.