[Discussion] How the pack lands on Claude 5-generation models

[lhoupert]

I've been using this pack for a few months (Claude Code, plus the Cursor links from install.sh). Last week I reviewed my whole ~/.claude setup against Anthropic's recent article on context engineering for Claude 5 models (https://claude.com/blog/the-new-rules-of-context-engineering-for-claude-5-generation-models), and this pack ended up being the biggest single thing I removed. I'd rather report the mechanism than just quietly drop it, because maybe something is fixable in the pack itself and not really about my setup

To be clear up front: I'm not saying the skills are wrong or that they don't help on ther models. I ran the review on one machine, and I don't yet have "before/after" data, I only removed things a few days ago.

The measurements below are from Claude against the repo at main

Unconditional cost. The 20 skill descriptions total ~4.7 KB, and they load into the system prompt of every session, including sessions that never touch code. The bodies are ~207 KB in total.

Pre-empting model judgement. 18 of the 20 carry a "Common Rationalizations" section, which exists to catch the model talking itself out of the prescribed process. The Anthropic article argues the opposite direction for this model generation — give rules, then let the model use judgement.

Overlap with host built-ins. On current Claude Code, code-review-and-quality, security-and-hardening, code-simplification and planning-and-task-breakdown each duplicate a built-in command (/code-review, /security-review, /simplify, /plan).

On the last point, #423 reports using-agent-skills stacking on GPT-5.6's built-in router. The same thing happens on Claude Code, which also routes skills natively. So that's the same mechanism on a second host, which I think supports the "don't load our router where the host already has one" note you suggested there.

My question is whether this is worth handling in the pack rather than each user discovering it? Would versioning skills per model generation work ? (e.g. a leaner variant for Claude 5-class models that keeps the process content but drops the rationalization/red-flag scaffolding.)

I asked more or less the same question about our own internal skills repo, so I'm genuinely unsure what the right answer is :-)

[addyosmani]

This is exactly the kind of report I want, thanks for writing up the mechanism instead of just quietly dropping it. Let me take each piece honestly, including where I think the framing is a little off.

The unconditional-cost number needs one correction. Only the descriptions load into every session, and yes, that ~4.7KB is a real always-on tax I should keep trimming. The ~207KB of bodies load on demand, one skill at a time, and only when a skill actually fires. That name-and-description split is the whole point of the design, so the honest always-on cost is the small number, not the big one. Still worth attacking, but it changes the size of the problem.

The rationalizations point is the sharp one, and I don't have a settled answer. You're right that "give rules, then let it judge" is the direction for this generation. My pushback is that those sections aren't uniform. On the discipline skills (test-driven, doubt-driven) they exist to stop the model talking itself out of a known-correct process under time pressure, which is a different thing from second-guessing a genuine judgment call. On the non-discipline skills they're closer to what you're describing and probably are net-negative on stronger models. The right way to settle that is to measure it, not argue it, which is what the Tier-3 behavioral evals are for, and it's also exactly what the drift data in #432 is offering. So my instinct is to trim the scaffolding where it demonstrably doesn't earn its tokens, per skill, rather than assume it's all dead weight.

The host built-in and router overlap is real, and it's the #423 problem generalized to a second host. "Don't load our router where the host already has one" was the note I landed on there, and you're right that it applies to Claude Code too. I want using-agent-skills to be host-aware rather than every user rediscovering it. That one I'd act on.

On per-generation variants: appealing, but a leaner fork doubles what I have to keep in sync, and drift like #432 shows is precisely what rots in the copy nobody is watching. My lean is host-aware routing plus measured, per-skill trimming of scaffolding that doesn't pull its weight, rather than a parallel Claude-5 catalog. I'm genuinely not certain though, so I'll leave this open and sit with it. Really useful report, and the fact that you asked the same question of your own internal repo is telling.