We have a tool at work that was entirely spun out of a prototype that itself was entirely driven by a single guy with an unlimited Claude budget. I only touch it on the edges, where I need to integrate it every so slightly into one of my systems. It's a crucial piece of software for the team, though. I've looked over the source, the thing that strikes me the most is the verbosity. Human software engineers would've been much more terse, so the thing has a fucking lot of code that's doing very little. They're trying to deal with that now, while also driving feature requests forward, and from the distance it looks like an unmanageable shit show.
In all fairness, LLMs try to write the code with all the safeguards imaginable, all exceptions handled and following the SDK/language/framework guidelines.
So a lot of this verbosity is good, even if we aren't used to it, because almost no one writes their code so exhaustively.
It really depends on the codebase. I remember ~10 years ago trying to learn about some of the big codebases that existed and what struke me was the verbosity. I don't remember if it was a PHP framework or a big JS library, but it was like 20+ lines of comment/explanation for each line of code. I was trying to follow the code flow and it was basically impossible, you could fit like 2-5 lines of actual code per screen.
Of course. Those 10 lines of extra code is that much more tokens every time your agents read and use that section of code. The more blot in the project to more tokens your agents will burn.
For one codebase I work in, LLM written code is noticeably and measurably (we have literally measured it for bugs, performance, etc.) better than what existed before. So the LLM code is a welcome change.
Do you measure anything about complexity or brevity? I'm currently working on importing some LLM code and it's 5-10x bigger than it should be for the actual needs.
One of my open questions is how much that increases over time. It could be that is a constant. But it also could be that, say, coding agents will infer false needs from the excess code/docs and elaborate further.
I've only personally utilized it for code _review_ and always ignored the suggested result code instead to understand the source of the issue and try to arrive at a resolution I find satisfactory for my mental model of the issue and solutions I'd prefer.
My take is that the main difference is the approach to problems the coding agents have. They optimize towards presuming a fully working invocation environment, yet checking everything anyway, and then rolling back any changes and re-testing that such changes worked. Which is great for normal types of software but incredibly tedious for anything aimed at less than a fully hands free automated environment.
In other words, AI is very awesome at scope creep of assigned problems and targeting the validation prompts baked into the review system.
I find this shocking though not unbelievable. Could you share how you measure this accurately? I'm interested in integrating such measurements into our services as well.
We’ve been tracking performance and bugs for years. Including commits those bugs were introduced in. So when LLM-generated code started working its way into our codebases, we have the before and after. And even comparing human generated code today with LLM-generated code today.
I find it unbelievable. I've seen the code LLMs write and it sucks compared to what a typical human produces. The only way an LLM is doing better than human programmers is if your human programmers were producing really terrible work.
I personally find it unbelievable that you have access to all of the public GitHub projects available and still think the typical developer writes good code.
What exactly do you mean when you say llm generated code? Are people prompting llms for changes and features without reviewing the code or iterating on it and then comparing that to what human writes? Because if so it's not surprising that you're getting worse results. Humans also write code through iteration. You can definitely get llms to write good code by enforcing guardrails and constraints through tooling and agent.md, and iterative reviews to nudge towards what you want. The first pass will look nothing like the committed code. I don't expect the llm to one shot anything.
Believe it or not, most programmers by definition are average. Hence, producing code better than them is not a hard feat to achieve for today's models.
You need a better harness. LLMs are significantly better at this than humans. Add it as a step in your LLM process and your documentation will always be up to date.
It encodes context it needs into doc comments and regular comments that make them in insufferable to read and so verbose because the comments contain info on other parts of the code base (that might have changed since that comment was written) but were useful to the model for its implementation. I have so far found no way to stop claude from doing this. It will sometimes do it with hand holding but the moment a task takes a slightly large amount of context its back to the word barf.
This is certainly a problem with out-of-the-box usage. But easily overcome.
Spend time updating your harness. This is not a hard problem to solve and there are many helpful guides out there.
We basically took our documented (human) SDLC process and applied it to the relevant harness hooks. Since our SDLC docs talked about what good comments looked like and what to avoid, we basically had the direction for the tools. Opus 5 did throw us a loop and increase the verbosity and decrease usefulness of the text, but refining our documentation cleared that hurdle within about a day.
'ai slop' is hard for me to complain about when it's better than 95% of code prior. The issue is more people trusting it too much to go too far, and you end up with lots of extraneous bullcrap you don't need. Doing too much and too big of a change is still a problem in the agentic world.
The field is already exploding in SLOC. If LLMs can generate 10x but only produce a 2x productivity boost in reasoning or understanding - each year -then neither humans or AI will be able to keep up with the mountains of slop that power everything from websites, telecom, banking, automotive, aerospace, industrial, medical, and renewables.
Guess we’ll need even more human software engineers to fix it.
> Guess we’ll need even more human software engineers to fix it.
This is what we've been doing for years before LLMs. Hiring a ton of devs to spit out code, which became another's burden. It just took longer and cost more.
Major critical software has always needed thousands of engineers to maintain million line codebases. LLMs can't reason across edge case failure modes, and better yet, LLMs generate compartmentalized modules which exponentially explodes the number of failure modes as LLM based codebases grow. Read how everyone here laments about Sol or Opus or GPT6 generates 10x the required number of lines
Used to be somewhat enjoyable. Nothing pleasant about digging around codebase that was heavily affected by the last 12-18 months of AI-ing.