I've been experimenting with all kinds of models lately - from "frontier", or should I say forbidden (hello, fable-5!) to "cheap". Here's an observation: When AI coding tools misbehave, the cause is almost never the model itself. It is the wrapper around the model. The harness mishandles edit operations. Prompts get mutated mid-session. The amount of context is too low and affecting the LLM in ways never intended (the "cheaper" the model, the more specific context you have to give). The model is fine. The plumbing is broken.
You have probably been in the equivalent organizational meeting. A team underperforms, an initiative stalls, an experiment fails. The diagnosis lands on the people. They lacked drive. They lacked skill. They were the wrong fit.
The diagnosis is almost never the plumbing. The reporting structure that delayed the signal. The lack of achievable goal-setting. The rushed beta features that left customers underwhelmed. The roadmap that changed three times. The decision-rights graph that made it nobody's job to push back. The senior leader that stepped in three times with "suggestions" that felt mandatory and were never met with scrutiny. The handoffs that lost half the context at each step.
The "model" is fine. The "harness" is broken.
Engineers learn this faster than organizational people do, partly because engineers can swap a harness, adjust the context level, and watch the AI coding output improve in real time. Organizations rarely get that feedback loop. Which is why the diagnosis keeps landing on the people, and the broken plumbing keeps surviving the postmortem.
Thoughts? Find me on Bluesky.