Add challenge 115: Fused QKV Projection with RoPE and KV Cache Update (Medium)

[claude]

Summary

Adds challenge 115 — Fused QKV Projection with RoPE and KV Cache Update (Medium), the prologue that every LLaMA-style attention layer runs on each decoding step:

1. One fused GEMM qkv = x @ W_qkv producing [B, (H_q + 2·H_kv)·D]
2. Split into grouped-query Q / K / V heads (H_kv ≤ H_q)
3. Rotate-half RoPE applied to Q and K only, using a precomputed cos_sin_cache[S_max, D] row selected by the per-sequence positions[b]
4. Scattered write of rotated K and raw V into K_cache / V_cache at slot positions[b], leaving every other cache slot untouched

Why this one

It is not a map operation — the solver has to reason about a skinny GEMM, the head-offset arithmetic of the fused QKV layout, a per-sequence gather of rotation coefficients, and a strided scatter into a multi-gigabyte cache that must not be copied or clobbered. The inout cache tensors make "touch only what you write" part of the correctness criterion.

No overlap with merged challenges (61 is standalone RoPE on a precomputed table, 84 is the MLP block, 96 reads an existing INT8 KV cache) or with any open PR.

Contents

• challenge.py — reference impl, signature, example test, 10 functional tests (edge sizes 1–4, powers of 2, non-powers of 2, zero input, realistic decode batches), performance test
• challenge.html — description, SVG dataflow diagram, worked example, constraints, references
• starter/ — all six frameworks (.cu, .pytorch.py, .triton.py, .jax.py, .cute.py, .mojo)

Validation

• pre-commit run --all-files passes
• A CUDA solution was submitted via scripts/run_challenge.py on a Tesla T4: all functional and performance tests pass (solution not committed)
• Performance test (B=64, d_model=4,096, H_q=32, H_kv=8, D=128, S_max=2,048) allocates ~1.18 GB, well within the 5× / 16 GB budget

🤖 Generated with Claude Code