Every Harness Has Its Quirks

You don’t use Claude, or GPT, or Gemini. You use a harness wrapped around them, and the harness is a different animal than the model inside.

Everyone keeps a private list

Run more than one agentic CLI and you’re keeping a list. Maybe in a scratch file, maybe just in your head. Not the model’s reasoning failures; those are loud and endlessly benchmarked. The small stuff. The things that make you mutter at the terminal and never quite get around to filing an issue.

Mine reads like a field journal from four expeditions. Kimi Code CLI: exit the session and the input box vanishes, so you’re typing blind into a smear of overlapping characters until you kill the pane. It crashes on loops. In headless mode it crawls, and you can’t tell whether it’s slow per token, slow per call, or just thinking three times as hard because nobody told it to stop. And @ can’t reliably find a file that’s sitting right there in the repo.

Codex barely belongs on a grievance list. It’s a good harness, fast and stable. One thing bites me: no per-project slash commands, so the repo-specific shortcuts I build into every other tool, I can’t build here. And when I do pick Claude over Codex, that’s the model talking, not the wrapper. Opus 4.8 edges GPT-5.5 for the work I do. It’s the one place in this whole essay where the model, not the harness, decides it for me.

Gemini’s the strange one. The model is brilliant, one of the smartest things I can put in a terminal, and I still won’t let it drive my code. I love it as a chat tool and open it every day. As a copilot it isn’t there yet, so it rides in the passenger seat. Claude I trust in the loop, and even Claude has its grievances: -p feels throttled, slower than the interactive session for reasons that never make the label, and every few weeks the nagging sense that it went dumb overnight.

Four tools, four catalogs, one shape of pain. Grant the Opus exception and none of the rest is the intelligence failing. It’s the input handling, the file search, the default flags, the headless plumbing. The friction sits one layer out, in the wrapper.

You touch the model a few times a session: an answer you read and accept or reject. You touch the harness on every keystroke.

The year the harness got a name

To its credit, 2026 is the year the industry stopped pretending the model was the whole story. “Harness engineering” got conference talks and a formula people actually repeat: Agent = Model + Harness. The model is the frozen part. The harness decides whether the frozen part does anything useful: its memory, tools, permissions, hooks, the whole runtime around the weights. The going line is that most of an agent’s real-world performance lives outside the model.

The cleanest proof is happening at Google right now. Gemini isn’t a weak model; by the numbers it’s one of the strongest there is. But its coding experience lagged so far behind its IQ that Google’s fix wasn’t a smarter model. It was a new wrapper. At I/O they retired the Gemini CLI and started moving everyone onto Antigravity. Same weights, different harness. Nobody decided Gemini needed a higher IQ. They decided it needed a better one wrapped around it. That’s the whole argument in a single product decision, and it’s why I keep the model in the passenger seat and still find myself impatient for the thing replacing the wrapper.

Here’s the part the whitepapers skip, though. They sell the harness as architecture: memory systems, tool contracts, observability, the stuff you design and put your name on. Real, but half the story. The other half has no diagram: it’s where the input box disappears and the default flag betrays you. The harness you build and the harness you suffer are the same harness. The industry only fell in love with the first one.

The day Claude “got dumber”

This was the most instructive thing that happened all year, and most people took the wrong lesson from it.

For about six weeks the threads piled up. Is it just me, or is Claude Code getting dumber? Theories about secret quantization. Accusations of a quiet bait-and-switch. The feeling was real. Something had changed.

Then Anthropic posted the postmortem, and the cause was almost boring. Three changes, none of them the model. March 4: the default reasoning effort got dropped from high to medium to fight latency, so it was simply thinking less. March 26: a caching change meant to clear stale thinking once instead cleared it every turn, so it went forgetful and repetitive. April 16: a verbosity instruction in the system prompt quietly hurt coding. Their words: “We never intentionally degrade our models… our API and inference layer were unaffected.” A default, a caching bug, a prompt tweak.

Don’t mock the people who cried nerf. Their gut was right; something had degraded. They just blamed the wrong layer, and it’s the easy layer to blame. The model has a name, a version number, a vendor to be mad at. The harness is invisible: undocumented defaults, prompt changes you’ll never see. When something feels off, you reach for the named suspect and walk straight past the silent one.

I do it too. I wrote up above that -p “feels throttled.” Maybe. More likely it’s a default or a context path behaving differently when no human is watching. “Throttled” is the named suspect; the honest version is a hypothesis I haven’t tested yet. Debug the tool the way you’d debug prod: go find what shipped, instead of posting that the vendor nerfed you this week.

Telling a genuinely worse model apart from a harness that moved is its own skill now. The most demanding users of the most-watched tool in the business spent six weeks certain they were watching the model rot. They were watching a flag flip.

Nobody benchmarks the papercuts

We measure models to death. SWE-bench, Terminal-Bench, dashboards tracking quality drift by the hour. Impressive, and aimed at the wrong target.

Gemini is the indictment. By the numbers it’s a triumph: SWE-bench Verified past eighty percent, top of every chart it appears on. And it’s still not a copilot I trust. The benchmark says best tool on my machine; my week put it in the passenger seat. Nothing on those charts measures that Kimi’s input box vanishes on exit, that @ whiffs on a file that’s right there, that you run /effort max to claw Claude back after a default regressed. There’s no eval for “the loop crashed and I lost my place.” That whole category of friction goes unmeasured, because it isn’t in the model.

And it’s the friction that sets your pace. A model two points higher whose harness eats your input and can’t resolve a path makes you slower, not faster. The leaderboard grades the engine. You’re bottlenecked by the transmission.

So harness fluency is concrete, even when it’s tacit: which loops crash, how to lay out a repo so @ resolves, which defaults to override on day one, what goes in AGENTS.md versus CLAUDE.md. It doesn’t carry over. Your Claude fluency buys you almost nothing the first day in Kimi. Which is exactly why it’s worth something. Everyone gets the same converging weights. Not everyone has put in the hours on the tool.

”That’s not real engineering”

The objection comes from good engineers: I shouldn’t have to memorize one vendor’s papercuts. Knowing that Kimi’s TUI breaks on exit isn’t a skill, it’s unpaid QA for a company that should fix its own tools. Fair. And wrong, the same way it’s been wrong before.

Typing code is commoditized. That’s the premise the whole industry runs on now. When the typing leaves, the leverage moves up a layer, to how well you drive the thing doing the typing. There was a wave of engineers who wouldn’t learn git because it was “just plumbing,” and another who wouldn’t touch the shell because it was “just glue.” The plumbing turned out to be the floor everything else stood on. They didn’t stay above it. They just got slower.

Harness fluency is this decade’s version of that, except the payoff is bigger, because the harness sits between you and every line you ship. A handicap there isn’t a handicap on one task. It’s a tax on all of them.

From the CTO chair

Everything above is “you” at the terminal. I also run an org, and from that seat the story gets bigger. The quirk isn’t one person’s tax, it’s that tax times every engineer who hits the same papercut alone.

The procurement instinct is to pick the winner: standardize on whatever model tops the chart, sign the deal, call it strategy. It feels responsible. It’s the expensive move, because it spends all your attention on the variable that’s converging and none on the one that compounds: how fluent your team is in the harness it works through every day.

Treat that fluency like infrastructure or watch it die with whoever holds it. The team whose CLAUDE.md, slash commands, hooks, and quirk-workarounds are checked into the repo and reviewed like code ships faster than the team where all of it is locked in three seniors’ heads. Same model, same tool, completely different pace.

Which makes constant tool-churn the org-level version of the nerf reflex: motion that feels like progress while resetting everyone to zero. Swap to the shinier CLI for a few benchmark points and you wipe months of accumulated fluency and re-bill onboarding from scratch.

So the job isn’t picking the smartest model; a spreadsheet can do that. It’s making fluency a shared asset that outlives any one engineer, and hiring for the part that survives a tool dying. Not “which model do you like,” but “can you tell a dumb model from a changed harness.” Ask anyone who’d gone deep on the Gemini CLI the week before Google retired it: the muscle memory evaporated, but the engineer who understood why a harness behaves the way it does was useful in the new one by lunch.

The model is what every competitor also gets to buy. Your team’s fluency in its harness is the thing they can’t.

The way out is the one we already built for the cloud

Fluency is the right answer for now. It can’t be the whole answer. Stop there and you’ve accepted the quirk tax as permanent. We’ve solved a problem this exact shape before.

Rewind ten years. Every cloud had its own console, its own CLI, its own lock-in, and “AWS fluency” was a real, stubbornly non-portable skill, the same shape as harness fluency today. We didn’t fix it by making everyone fluent in three clouds. We built Terraform. Declare what you want once, target whichever provider, and the skill moves up a level: from memorizing one vendor’s console to describing intent. Lock-in stopped being fate.

The same move is open for harnesses, and it’s overdue. Be harness-agnostic the way you learned to be cloud-agnostic. Not perfectly; Terraform still leaks provider-specific resources to this day. But enough that swapping the tool underneath your work is a config change instead of a re-education.

It can work because the quirks split into two kinds. The mechanical ones are interface friction: claude -p versus codex exec, the flag differences, CLAUDE.md versus AGENTS.md, whether your skill packs port to the next tool. Interface friction is precisely what an abstraction layer dissolves. The behavioral ones don’t abstract away: the TUI eating your input, the flipped default, whether the model is sharp this week. So agnosticism doesn’t replace fluency. It splits the work: abstract the mechanical, stay fluent on the behavioral.

I’ll put money where my mouth is, because pointing at a direction is cheap. I built a small router, par, that gives my scripts one stable prompt interface and forwards it to whatever local harness I aim it at, translating the call into that tool’s native command. It’s nowhere near a Terraform. It’s an early and deliberately small bet on one idea: a script should say what it wants, not which CLI happens to be in fashion this quarter. When the Gemini CLI shuts off for Antigravity in two weeks, the wager is that I rewrite one adapter instead of every pipeline I own.

That’s the actual exit. Fluency is how you survive the harness today. Abstraction is how you stop being its hostage tomorrow.

What to actually do

Keep the journal. The scratch file of grievances is the most useful document in your workflow. Write the quirk down the moment it bites: the loop that crashed, the flag that fixed it. It’s the manual the vendor didn’t ship.

Then the boring stuff. Pin your versions so the ground stops moving without your say-so. Read the changelogs and the postmortems; Anthropic told everyone exactly what broke and when, and most of the people raging in the threads never read it. Override the defaults; they’re tuned for latency demos, not your work. Go deep enough in one harness to be fast today, but don’t wire yourself in so tight that the day it’s retired costs you a re-education instead of an adapter.

And the next time it “gets dumb,” file the bug against yourself first. What changed, what version, what default, what shipped. The urge to broadcast a nerf is a tell that you’ve stopped treating the harness as something you can reason about.

The teams pulling ahead in 2026 aren’t on the highest-benchmark model. They know their harness cold. They reach past the broken input box without thinking, they feel a regression in a day instead of a month, and they’re quietly building the abstractions that will let them walk away from it when they need to. Every harness has its quirks. Learning them is the craft. Not being trapped by them is the next one.