It has always been non-deterministic but we relied on low level engineers who knew the dark magicks to keep the horrors at bay.
Bit flips in memory are super common. Even CPUs sometimes output the wrong answer for calculations because of random chance. Network errors are common, at scale you'll see data corruption across a LAN often enough that you'll quickly implement application level retries because somehow the network level stuff still lets errors through.
Some memory chips are slightly out of timing spec. This manifests itself as random crashes, maybe one every few weeks. You need really damn good telemetry to even figure out what is going on.
Compilers do indeed have bugs. Native developers working in old hairy code bases will confirm, often with stories of weeks spent debugging what the hell was going on before someone figured out the compiler was outputting incorrect code.
It is just that the randomness has been so rare, or the effects so minor, that it has all been, mostly, an inconvenience. It worries people working in aviation or medical equipment, but otherwise people accept the need for an occasional reboot or they don't worry about a few pixels in a rendered frame being the wrong color.
LLMs are uncertainty amplifiers. Accept a lot of randomness and in return you get a tool that was pure sci-fi bullshit 10 years ago. Hell when reading science fiction now days I am literally going "well we have that now, and that, oh yeah we got that working, and I think I just saw a paper on that last week."
With the old way of doing things you could spend energy to reduce errors, and balance that against the entropy of you environment/new features/whatever at a rate appropriate for your problem.
It's not obvious if that's the case with llm based development. Of course you could 'use llms until things get crazy then stop' but that doesn't seem part of the zeitgeist.
> It's not obvious if that's the case with llm based development. Of course you could 'use llms until things get crazy then stop' but that doesn't seem part of the zeitgeist.
Harnesses are coming online now that are designed to reduce failure rates and improve code quality. Systems that designate sub-agents that handle specific tasks, that put quality gates in place, that enforce code quality checks.
One system I saw (sadly not open source yet) spends ~70% of tokens on review and quality. I'll admit the current business model of Anthropic/OpenAI would be very unfriendly to that way of working. There is going to be some conflict popping up there. Maybe open weight models will save us, maybe not.
If Moore's Law had iterated once or twice more we wouldn't be having this conversation. We'd all be running open weight models on our 64GB+ VRAM video cards at home and most of these discussions would be moot. AI company valuations would be a fraction of what they are.
> It has always been non-deterministic but we relied on low level engineers who knew the dark magicks to keep the horrors at bay.
This is a disingenuous comparison.
First of all, what you're talking about is nondeterminism at the hardware level, subverting the software, which is, on an ideal/theoretical computer, fully deterministic (except in ways that we specifically tell it not to be, through the use of PRNGs or real entropy sources).
Second of all, the frequency with which traditional programs are nondeterministic in this manner is multiple orders of magnitude less than the frequency of nondeterminism in LLMs. (Frankly, I'd put that latter number at 1.)
This is part of a class of bullshit and weaselly replies that I've seen attempting to defend LLMs over the years, where the LLMs' fundamental characteristics are downplayed because whatever they're being compared to occasionally exhibits some similar behavior—regardless of the fact that it's less frequent, more predictable, and more easily mitigated.
> First of all, what you're talking about is nondeterminism at the hardware level, subverting the software, which is, on an ideal/theoretical computer, fully deterministic (except in ways that we specifically tell it not to be, through the use of PRNGs or real entropy sources).
Malloc and free were never deterministic outside of the simplest systems.
The second we accepted OS preemption we gave up deterministic performance.
Good teams freeze their build tools at a specific version because even minor revs of compilers can change behavior.
I've used way too many schema generator tools that I'd describe as "wishfully deterministic".
Heuristics have been used for years in computer science, resulting in surprising behavior. My point is that if we ramp up the rate of WTF we are willing to tolerate, the power of the systems we can build increases drastically.
> Second of all, the frequency with which traditional programs are nondeterministic in this manner is multiple orders of magnitude less than the frequency of nondeterminism in LLMs. (Frankly, I'd put that latter number at 1.)
Building a RAG lookup system that takes in questions from the user, looks up answers in a doc, and returns results, can be built with reliability damn near approaching 99.99%.
I have seen code generation harnesses that also dramatically reduce non-determinism of LLM generated code, but that will continue to be a hard problem.
My phone camera applies non-deterministic optimizations to images I take, and has done so for years now.
GPS is non-deterministic (noisy), we smooth over the issues. GPS routing is also iffy, but again we smooth over the issues.
The question is can useful products be made with a technology. You can shove enough guardrails on an LLM interface to make it useful. That much is clear. I derive massive value from LLMs and other transformer based systems literally everyday. From the modern speech transcription systems, that are damn near magic compare to what we had a few years back, to image recognition, to natural language interfaces to search over company documents.
If we completely discard coding agents, LLMs are still an insanely impactful technology.
Those guardrails add costs, and latency. For some scenarios that is fine, but for others it isn't. Chat bot support agents implemented by the lowest bidder don't have any attempt at guardrails. Better systems are better built.
I agree that current LLMs all suffer from the problem that the control messages are intermixed with data, that is a crappy problem that the industry has known is a bad pattern for literally decades (since the 70s, 80s?). It seems like an intractable flaw in the systems.
But that doesn't make the system unusable any more than the thousand other protocols suffering from the same flaw are unusable.
The single best example is for this discussion is Superscalar out-of-order execution which can't be used in aerospace, medical devices, and industrial control systems, or you need to guarantee that code finishes within a certain time, because technically it isn't deterministic.
Neither is stochastic gradient descent which is the cause of the LLM problem. Nor is UDP, the network protocol that powers video calls, live streaming, and online gaming.
I sympathize. But it seems so weird to me to be a Linux open source adopter, who is also so am conservative. Heck yes KDE, sway, Niri, and so many other new desktops are so so so nice. I can't imagine staying in the old world.
But who should AMD for example target? It seems obvious to me. Personally I think if you are trying to make tools like ai acceleration software, you should 100% be focused on the alpha geeks. The people who want bleeding edge new stuff, the people who will take that and roll with it and expand your ecosystem are almost entirely the early adopters. The progressives are the social tastemakers, are the point on the adoption curve where cool and good happens.
It's infuriating watching AMD bungle their chances by targeting people interested in ancient technology, for their incredibly advanced powerful rich new stack. What are you doing?!?! There needs to be a starting point for the alpha geeks, those who are willing.
> But it seems so weird to me to be a Linux open source adopter, who is also so am conservative.
I used to be a tech enthusiast who installed the latest bleeding edge everything.
Now days I need a stable machine that lets me work. Sometimes my work is bleeding edge, but I want my machine to be boring and stable.
I can't run the latest open source models if I can't get my machine to even boot.
Also some of the latest AI stuff (desktop control, full screen recording, etc) doesn't support wayland. E.g. https://pieces.app/ prefer x11 due to the simpler security model.
TheRock is so close but there just doesn't seem to be an eye on the ball for actually making this usable. This stack of issues blocking even vaguely modern systems has been sitting untouched for months.
Cable TV begs to differ. I grew up working poor and plenty of people around me dumped a lot of money into cable TV subscriptions, and $120 back in the late 90s is $240 now.
Computer costs keep collapsing. Image and audio generation is turned out to be less computer intensive than text (lol).
First company to launch 24/7 customized streaming AI slop wins!
I think the poster was saying giving away the models for $200 isn't sustainable for the provider, not that a user won't pay $200 for the latest and greatest models.
That is ultimately where it is headed and has been headed for over 100 years now.
The question is when will we get there.
If the answer is tomorrow, money means nothing and none of these investments matter. If the answer is 30 years, well lots of money to be made up until the inflection point of machines being able to design, build, and repair themselves.
You heard AI powered scissors and thought that's where we're heading? I think you'd have to be totally divorced from the average person to believe this.
Meanwhile people are still begging car manufacturers to stop locking their glove box behind a touch screen. Or how about a TV that isn't loaded with crappy software that makes it unusable after 2 years. There's a reason we don't put tech in everything.
Well, fortunately (from a certain point of view), it doesn't really matter what people beg for as long as they need the thing anyway and you and your competitors all agree not to give them what they want.
Linux has Snap, and Flatpak, and sometimes an app works with one or not the other.
It is the worst of both worlds.
I especially love how flatpak has its own version of graphics drivers, which results in my browser flatpak updating to a version that has a mismatch between my system drivers, flatpak, and the browser.
But say the Flatpak strategy were adopted fully, would it solve this? I've never noticed the permission difference between using (Snap or Flatpak) vs not, but I need to try it more.
I got an android based epaper device and installed Libby on it. Overall it is a much better experience than doing the library to Kindle dance. I can just browse my local library's ebook collection on device and immediately open them in one tap.
Battery life standby time isn't nearly as good, but being able to also read Notion pages, review full PDFs, and other benefits from having an actual tablet, make the battery life sacrifice worth it.
If that is true, of which I remain highly skeptical, then it implies that books are wildly inefficient to produce.
What on earth are all the middlemen between book being authored and it being sold to a customer that add so much overhead that the cost of printing and logistics disappears in the noise???
> If that is true, of which I remain highly skeptical, then it implies that books are wildly inefficient to produce.
It just means that publishers are really good at manufacturing physical goods. They've been doing it for several hundred years so no big surprise there.
Books don't sell in large quantities. The economics of scale for the publishes for labor aren't there.
No one is getting rich off of fiction publishing except for the rare break out author. Publishers go out of business (or get acquired) all the time because they are constantly one step away from being insolvent.
This is also why the industry has massively consolidated.
I highly suggest reading breakdowns of the finances of publishing books, it is an interesting field that is incredibly different than how we are used to seeing numbers work in software.
The middlemen are giving your book some (still probably rather small) chance of being bought in significant numbers. If you just want a big stack of books and don't care if anyone buys them, they're not especially expensive to produce.
When you consider that different ebooks and different font selection can result in lines and pages breaking at any random place, ebooks may actually be more expensive to produce.
Don't think I've ever read a properly produced ebook. Page breaks fall wherever and formatting is dictated more by my size/border/etc choices than by whomever "produced" then book.
Nevertheless automatic typesetting and formatting have existed for decades! TeX and LaTeX are ancient and produce better looking results than any book I've ever read on any of my ereaders, and those aren't the only tools in this space.
Whatever people are paying for such "production" seems wasted.
I converted ebooks into PDFs specifically formatted for my reader size and typeset in the fonts I like. It had proper kerning, hyphenation, widow/orphan control, drop capitals, etc.
However that PDF is not reflow-able (or changeable in any way) once it's on the device, and that's not what people are buying ebook readers for.
Bit flips in memory are super common. Even CPUs sometimes output the wrong answer for calculations because of random chance. Network errors are common, at scale you'll see data corruption across a LAN often enough that you'll quickly implement application level retries because somehow the network level stuff still lets errors through.
Some memory chips are slightly out of timing spec. This manifests itself as random crashes, maybe one every few weeks. You need really damn good telemetry to even figure out what is going on.
Compilers do indeed have bugs. Native developers working in old hairy code bases will confirm, often with stories of weeks spent debugging what the hell was going on before someone figured out the compiler was outputting incorrect code.
It is just that the randomness has been so rare, or the effects so minor, that it has all been, mostly, an inconvenience. It worries people working in aviation or medical equipment, but otherwise people accept the need for an occasional reboot or they don't worry about a few pixels in a rendered frame being the wrong color.
LLMs are uncertainty amplifiers. Accept a lot of randomness and in return you get a tool that was pure sci-fi bullshit 10 years ago. Hell when reading science fiction now days I am literally going "well we have that now, and that, oh yeah we got that working, and I think I just saw a paper on that last week."
reply