Skip to content

Update simplew tuned - #1105

Merged
MDA2AV merged 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned
Aug 13, 2026
Merged

Update simplew tuned#1105
MDA2AV merged 2 commits into
MDA2AV:mainfrom
stratdev3:update-simplew-tuned

Conversation

@stratdev3

Copy link
Copy Markdown
Contributor

Description

update ioxide from v0.0.17 to v0.4.186


PR Commands — comment on this PR to trigger (requires collaborator approval):

Command Description
/benchmark -f <framework> Run every test the framework subscribes to
/benchmark -f <framework> -t <test> Run one test only
/benchmark -f <framework> --save Run and save results (updates the leaderboard on merge)
/benchmark -f <framework> -t <test> --save Run one test and save results
/benchmark -f <framework> --compare <other> Measure the deltas against another framework instead of this one
/benchmark-multiple -f <fw1>,<fw2>,... Benchmark several frameworks in one run — takes -t and --save too; saved results land in a single commit
/benchmark-multiple --save No -f needed: benchmark and save every framework the PR touches
/benchmark-test -t <test> Benchmark all enabled frameworks subscribed to <test> and save the results

For /benchmark, always specify -f <framework>; the flags combine in any order. Results come back as a comment with a per-profile table of RPS, p99, CPU and memory — one table per framework on multi runs. A new benchmark comment while a run is in flight queues behind it (one deep) instead of cancelling it. For multi-framework PRs (dependency bumps, same-language refactors) prefer /benchmark-multiple, which runs everything in a single job and commits all saved results together, so no run overwrites another. --compare works on single-framework runs only.

What the deltas are measured against. By default, this framework's own results published on main - answering "did this change help?". When you are tuning a variant or a successor entry, --compare re-bases them on another entry instead:

/benchmark -f genhttp-11 --compare genhttp

The reply states which baseline it used, and profiles the other framework does not run show n/a rather than a delta.


Run benchmarks locally

You can validate and benchmark your framework locally with the lite script — no CPU pinning, fixed connection counts, all load generators run in Docker.

./scripts/validate.sh <framework>
./scripts/benchmark-lite.sh <framework> baseline
./scripts/benchmark-lite.sh --load-threads 4 <framework>

Requirements: Docker Engine on Linux. Load generators (gcannon, h2load, h2load-h3, wrk, ghz) are built as self-contained Docker images on first run.

@stratdev3

Copy link
Copy Markdown
Contributor Author

/benchmark -f simplew-tuned

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Results

Framework: simplew-tuned | Test: all tests

Test Conn RPS CPU Mem Δ RPS Δ Mem
baseline 512 2,876,278 6137.9% 1.4GiB +8.5% +139.3%
baseline 4096 3,050,575 6001.0% 1.5GiB +12.7% +60.3%
pipelined 512 15,605,107 5948.1% 1.4GiB +239941.6% +247.1%
pipelined 4096 15,533,478 6260.8% 1.6GiB +22927.6% +172.6%
limited-conn 512 2,013,595 6029.0% 1.4GiB +4.1% +162.1%
limited-conn 4096 2,228,213 6114.0% 1.6GiB +3.3% +81.6%
json 4096 1,001,207 6144.6% 1.7GiB +2.4% +87.2%
json-comp 512 535,330 5853.6% 1.4GiB -0.6% +98.0%
json-comp 4096 541,277 6142.1% 1.7GiB +3.5% +70.0%
json-comp 16384 526,785 6045.3% 2.1GiB +4.7% +40.0%
json-tls 4096 813,414 6242.6% 2.5GiB +15.7% +219.2%
upload 32 1,959 1932.2% 8.4GiB NEW +1737.9%
upload 256 2,127 4169.7% 8.5GiB NEW +1654.8%
api-4 256 46,038 396.4% 212MiB -5.5% +34.2%
api-16 1024 155,294 1672.6% 625MiB -8.7% +42.0%
static 1024 285,381 6013.7% 1.6GiB +3.1% +103.5%
static 4096 279,786 5745.7% 2.1GiB +2.0% +50.0%
static 6800 277,468 5942.5% 2.6GiB +2.6% +44.4%
async-db 1024 166,515 4343.4% 1.6GiB +32.4% +125.7%
crud 4096 407,426 4965.4% 1.8GiB -2.4% +50.0%
echo-ws 512 1,170,877 5657.7% 1.3GiB -56.3% +141.2%
echo-ws 4096 1,954,903 6293.2% 1.6GiB -26.6% +73.9%
echo-ws 16384 1,750,397 5975.9% 2.3GiB -22.3% +9.5%
echo-ws-pipeline 512 2,123,021 6457.5% 1.4GiB -32.2% +163.0%
echo-ws-pipeline 4096 1,949,670 6310.2% 1.6GiB -39.8% +45.5%
echo-ws-pipeline 16384 1,959,726 6011.7% 2.3GiB -33.8% +21.1%
Full log
  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.86ms   2.86ms   6.49ms   19.30ms   49.90ms

  10615107 frames sent in 5.00s, 10615107 frames received
  Throughput: 2.12M req/s
  Bandwidth:  14.17MB/s
  WS upgrades: 512
  WS frames:   10615107
  Latency samples: 10615071 / 10615107 responses (100.0%)
[info] CPU 6457.5% | Mem 1.4GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     512 (8/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   3.88ms   2.87ms   6.48ms   20.80ms   47.80ms

  10572105 frames sent in 5.00s, 10572104 frames received
  Throughput: 2.11M req/s
  Bandwidth:  14.11MB/s
  WS upgrades: 512
  WS frames:   10572104
  Latency samples: 10572050 / 10572104 responses (100.0%)
[info] CPU 6396.2% | Mem 1.4GiB

=== Best: 2123021 req/s (CPU: 6457.5%, Mem: 1.4GiB) ===
[info] saved results/echo-ws-pipeline/512/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 4096c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   36.04ms   29.20ms   59.90ms   126.60ms   194.80ms

  8992809 frames sent in 5.00s, 8930345 frames received
  Throughput: 1.79M req/s
  Bandwidth:  12.04MB/s
  WS upgrades: 4096
  WS frames:   8930345
  Latency samples: 8930345 / 8930345 responses (100.0%)
[info] CPU 5633.8% | Mem 1.5GiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   33.23ms   29.10ms   52.20ms   89.90ms   125.70ms

  9808706 frames sent in 5.00s, 9748354 frames received
  Throughput: 1.95M req/s
  Bandwidth:  13.13MB/s
  WS upgrades: 4094
  WS frames:   9748354
  Latency samples: 9748354 / 9748354 responses (100.0%)
[info] CPU 6310.2% | Mem 1.6GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     4096 (64/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   33.52ms   29.30ms   52.70ms   87.00ms   124.00ms

  9772097 frames sent in 5.00s, 9715697 frames received
  Throughput: 1.94M req/s
  Bandwidth:  13.08MB/s
  WS upgrades: 4095
  WS frames:   9715697
  Latency samples: 9715697 / 9715697 responses (100.0%)
[info] CPU 6177.2% | Mem 1.6GiB

=== Best: 1949670 req/s (CPU: 6310.2%, Mem: 1.6GiB) ===
[info] saved results/echo-ws-pipeline/4096/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned

==============================================
=== simplew-tuned / echo-ws-pipeline / 16384c (tool=gcannon) ===
==============================================
[info] waiting for server...
[info] server ready

[run 1/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   167.81ms   148.60ms   253.50ms   511.20ms   697.60ms

  7543905 frames sent in 5.00s, 7281761 frames received
  Throughput: 1.46M req/s
  Bandwidth:  10.24MB/s
  WS upgrades: 16384
  WS frames:   7281761
  Latency samples: 7281761 / 7281761 responses (100.0%)
[info] CPU 4962.6% | Mem 1.9GiB

[run 2/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   131.64ms   128.00ms   181.70ms   237.80ms   307.10ms

  9752768 frames sent in 5.00s, 9490624 frames received
  Throughput: 1.90M req/s
  Bandwidth:  13.19MB/s
  WS upgrades: 16384
  WS frames:   9490624
  Latency samples: 9490624 / 9490624 responses (100.0%)
[info] CPU 6244.1% | Mem 2.1GiB

[run 3/3]
gcannon v0.5.3 [WS]
  Target:    localhost:8080/ws
  Threads:   64
  Conns:     16384 (256/thread)
  Pipeline:  16
  Req/conn:  unlimited (keep-alive)
  Expected:  200
  Duration:  5s


  Thread Stats   Avg      p50      p90      p99    p99.9
    Latency   130.77ms   125.30ms   185.30ms   241.50ms   307.20ms

  10060774 frames sent in 5.00s, 9798630 frames received
  Throughput: 1.96M req/s
  Bandwidth:  13.60MB/s
  WS upgrades: 16384
  WS frames:   9798630
  Latency samples: 9798630 / 9798630 responses (100.0%)
[info] CPU 6011.7% | Mem 2.3GiB

=== Best: 1959726 req/s (CPU: 6011.7%, Mem: 2.3GiB) ===
[info] saved results/echo-ws-pipeline/16384/simplew-tuned.json
httparena-bench-simplew-tuned
httparena-bench-simplew-tuned
[info] skip: simplew-tuned does not subscribe to echo-ws-limited
[info] rebuilding site/data/*.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/frameworks.json
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/results/simplew-tuned.json - 26 new, 26 total
[updated] /home/diogo/actions-runner/_work/HttpArena/HttpArena/site/data/current.json
[info] done
httparena-postgres
httparena-redis
[info] restoring loopback MTU to 65536

@MDA2AV

MDA2AV commented Aug 12, 2026

Copy link
Copy Markdown
Owner

/benchmark -f simplew-tuned --save

@github-actions

Copy link
Copy Markdown
Contributor

👋 Benchmark request received. A collaborator will review and approve the run.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ /benchmark --save cannot start: main has diverged and cannot be auto-merged into this branch. Please merge or rebase main manually, push, and re-run /benchmark --save.

@stratdev3

Copy link
Copy Markdown
Contributor Author

thanks you can merge that PR

@MDA2AV
MDA2AV merged commit 5d803e0 into MDA2AV:main Aug 13, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants