It's useless to talk about models and harnesses without context and method. Depending on how you use the model and what the model is used for, experience may vary drastically. Also, different models with different harnesses require different approaches.
I've been using https://gitlab.com/gabriel.chamon/orisun which is my own simplified methodology, for coding web apps in python and elixir and have been very successful using qwen3.6 27b Q4 locally with help of larger models for architecture, so I get very suspicious when people talk how useless larger models are. They are either using it for a domain that models don't perform well or just not using it right.
I'm not sure about "useless" but from my experience agentic coding leads to death by a thousand cuts for all projects I've seen so far. Small decisions missed in a codebase that leads to degradation in correctness, reliability and performance. At some point it only takes one engineer to be careless, others skipping PR because they are AI generated...
I got into a bit of an argument a while back when I used the word "crass" to describe some of the code decisions I've seen Claude make (in someone else's project that I have to work with).
But it is how I feel and it feels like the right word for the job. Because as you say, good code projects start out with good decisions.
It's like when you see a CAD design with a sequence of features that exist only to fix problems caused by starting from the wrong principles or the wrong baseline.
Sure the resulting part may end up identical as a solid for that specific need, but it could have been done in a way that was more robust, simple, easier to understand and modify, and where the design doesn't break in an unexpected way due to a small change of an early measurement.
(CAD has made my instincts much more visible to me)
Exactly, and even when humans make bad decisions there is some friction, I feel that llm's don't have/notice that friction, they just bulldoze without caring about anything else.
Not my experience. It does depend on the scale of the project, but for tens of thousands LOC, it's really good at taking instructions for architectural redesign.
LLM will take stuff at face value. If you tell it it's good, it'll gladly abide. Similarly if say it's crap it'll so the complete opposite.
Yes, it's good at taking instructions for architectural redesign, but when it's not doing an actual architectural redesign task, it will be heavily biased towards the status-quo, despite AGENTS.md and prompts trying to steer it.
My latest experience was towards a C++ to Rust migration and it failed miserably because the existing codebase enforced patterns I didn't want.
Ah yes, I understand, but that's expected and actually desirable, it's preferable than having inconsistent project patterns across the project introduced by multiple LLMs. Imagine the mayhem.
It's important to do a first pass cleaning up and preparing code before unleashing agents onto it, similar to how Working Effectively with Legacy Code advocates approaching consolidated codebases. After that the agent will actually pick up the new pattern and start propagating that to the rest of the code.
I believe it’s desirable as a default, but not when the prompting/steering asks not to.
> It's important to do a first pass cleaning up and preparing code before unleashing agents onto it
That’s a good idea in general, but the whole point of the rewrite was that Fable wasn’t capable enough of fixing the old codebase which we were using as a reference.
Another problem I had in another project was with having “sample” code purely written for testing being used as the architectural guidance, despite comments and repeated memories asking not to.
In the end, by the way, the solution was to simply blocking access to specific files, in the Claude settings.json.
But still Claude tried to cheat by running ‘cat file.cpp` a few times, so…
All of which you had with only humans in the loop. Catalogue problems so they become technical debt and tackle them periodically. Seems to me like this is less of an AI problem and more of bad management.
It's a solvable problem if you're willing to throw more tokens at it. Frontier models have gotten very good at cleaning up their own messes. You just need the right skills/loops, and to stick to models that consistently follow instructions (i.e. GPT-5.5/GPT-5.6-Sol).
I've been using https://gitlab.com/gabriel.chamon/orisun which is my own simplified methodology, for coding web apps in python and elixir and have been very successful using qwen3.6 27b Q4 locally with help of larger models for architecture, so I get very suspicious when people talk how useless larger models are. They are either using it for a domain that models don't perform well or just not using it right.