Washington | 28°C (broken clouds)
The Demotion Ladder: Lessons From a Year of Taming Claude Code

The Demotion Ladder – A Year of Governing Claude‑Generated Code

A lead engineer recounts fourteen months of wrestling with Claude‑Code, uncovering why massive rulebooks fail, how compaction eats detail, and why “never do X” quickly fades.

When I first started feeding Claude the style guide I kept for my team, I imagined it would behave like a checklist. I’d spot a typo, add a rule, and the next run would be a shade cleaner. In the early days that illusion held – a couple of rules, a couple of wins, and a tiny smile each time the model copied my conventions.

Fast forward to May 2025: I was running Claude against a live product with about a thousand endpoints, auto‑generated API contracts, Excel and PDF exports, a notification system, page templates and the inevitable edge‑case workarounds. Fourteen months later the “rules doc” had exploded into eighteen separate files, each 250–750 lines long, topping five thousand lines of prose. And that’s before you even count the architecture diagrams or process notes.

To test the beast, I handed it a fresh feature slice – a CRUD screen with a table, a form, details view and a handful of actions. Everything the feature needed already lived in the repo as examples, and the style guide had entries for every relevant convention. Claude spat out a bare‑bones MVP. It missed the endpoint hierarchy, ignored the generated frontend client, applied the UI theme incorrectly, and refused to reuse any of the existing page templates or components. The mapper was mis‑wired, handlers were replaced by hacky shortcuts, and the number of endpoints was off. Tests were half‑missing, a few broke outright, and the build failed. It felt like ten junior devs had each reinvented a piece of the puzzle without ever looking at the rest of the code.

If I dumped those five thousand lines of rules on a human junior, they’d probably drown as well. I was hoping a machine, even a not‑quite‑human‑brain, could at least keep its head above water.

Anthropic’s own docs are brutally honest: “Bloated CLAUDE.md files make the model ignore your real instructions. Prune ruthlessly.” The common advice now is to turn prose into deterministic hooks and to write tests instead of long paragraphs. I nodded along, but the problem wasn’t whether prose was weak – it was figuring out which rules needed to become hooks, when, and what price that would exact.

My early theory was simple: the massive rule corpus was competing with the task for context slots. If Claude could load the whole set without a warning about token overflow, I assumed everything was fine. That intuition felt right until a May 2026 pre‑print shook it. The study ran 1,650 Claude‑Code sessions across two codebases, tweaking config size, instruction placement, and internal contradictions. The only thing that measurably shifted compliance was the within‑session effect – each extra function Claude generated lowered compliance odds by about 5.6 %. Their biggest config was 500 lines; my rulebase was ten times that. The study’s null result suggested sheer size wasn’t the culprit, forcing me to look elsewhere.

What emerged was three overlapping realities.

1. Rules are scoped per unit of work. When Claude tackles a narrow bug, only a handful of relevant rules matter and they stay in context. A vertical slice, however, drags in naming conventions, contract hierarchies, theme constraints, and test policies all at once. I built a 68‑line hook that inspected the prompt and pulled only the docs matching the work type (backend, frontend, or testing). It trimmed the noise but didn’t solve the core issue – Claude still needed every rule that applied to that slice, and the cost shifted to stitching everything back together.

2. Compaction robs detail. On a sizable feature Claude compacts its output 20–50 times before finishing. Every compaction step discards bits of the original policy. I found myself sprinkling an extra 50 k tokens of reminders after each round – “don’t forget those five docs!” A June 2026 pre‑print measured the same phenomenon over 1,323 episodes: policy violations rose from 0 % with full context to 30 % after one compaction, and as high as 59 % on some models. When the constraint survived the summary, violation rates stayed at zero.

3. Prohibitions decay faster than mandates. A later pre‑print (4,416 trials, 12 models) showed “never do X” compliance fell from 73 % at turn 5 to a bleak 33 % by turn 16, while “always do X” held steady at 100 %. Almost every rule I’d written was a prohibition – “don’t name variables this way”, “never omit a test”, etc. In hindsight I was fighting an asymmetry that favors positive commands.

Two quick caveats: most of the cited studies are pre‑prints and haven’t been peer‑reviewed, so treat them as promising data points rather than gospel. And my own year‑long saga isn’t a controlled experiment – the model saw eight major releases, the harness gained new primitives, the codebase accumulated reusable examples, and my own prompt‑crafting skills improved throughout.

Bottom line? A sprawling prose rulebook isn’t the silver bullet. You need a lean set of deterministic hooks, an awareness that every extra function drags compliance down, and a bias toward “always do” statements. Otherwise, you’ll find yourself on a demotion ladder, watching Claude’s output slide farther from the standards you painstakingly wrote.

Comments 0
Please login to post a comment. Login
No approved comments yet.

Editorial note: Nishadil may use AI assistance for news drafting and formatting. Readers can report issues from this page, and material corrections are reviewed under our editorial standards.