Hacker Newsnew | past | comments | ask | show | jobs | submit | djoldman's commentslogin

TFA reasonably reduces to:

First, ATMs increased the demand for bank branches, which more than made up for the decrease in tellers per branch.

Second, mobile banking decreased the demand for physical branches.


There are ATMs not attached to bank branches. They could have replaced the branches with ATMs before. (I do wonder what bank tellers are doing these days. I mean actual tellers, not investment advisors and jobs like that.)

They are handling in-person transactions, usually deposits (many who deposit checks manually still don't know how to use the app to do so, or if the branch has an ATM that does deposits).

They are the only way to get non-20 cash in many areas; the ATMs that can dispense other bills are quite rare. And if you want $100 in ones you're going inside.


They're basically bank receptionists for old people who will type details into the same system that the general public has access to. They also handle cash for small businesses (I worked in a cafe during university and we'd regularly have to do runs into town to deposit rolls of bills and get more change to float the till)

If that's all you think tellers are then you're missing out on a lot of opportunities.

They are the first line of human-to-human contact with customers. They are able to sell new services or upsell existing services to customers, especially with the customer's data right in front of them. A new pleasant conversation plus "Oh by the way, did you know that you could get service ABC that would help you?" is something that an LLM or ATM can't do reliably.

There's a tremendous amount of opportunity available with well-trained tellers.


For folks reasoning through the "paradox," this may be helpful:

https://arxiv.org/pdf/0904.2540

Abstract:

> ...We show that the conflicting recommendations in Newcomb’s scenario use different Bayes nets to relate your choice and the algorithm’s prediction. These two Bayes nets are incompatible. This resolves the paradox: the reason there appears to be two conflicting recommendations is that the specification of the underlying Bayes net is open to two, conflicting interpretations...




Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully."

Or the long version: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness."

Or the short versions: "Skippetyboop," "plipnikop," and "zingybang."


I've largely avoided using the term "AI" to refer to the current LLM and generative technology because it's loaded with too much ambiguity and glosses over the problems with those technologies in the context of conversations around it.

Same, but I have used the term “generative AI” to describe generative models. Never the naked “AI” though (except in conversations with friends where the difference is pedantic because they’re not subject matter experts).

"AI" implies intelligence, which is nowhere to be found. "Text generators" is the best descriptive term.

"applied statistics"

>Anytime I see "Artificial General Intelligence," "AGI," "ASI," etc., I mentally replace it with "something no one has defined meaningfully."

There are lots of meaningful definitions, the people saying we haven't reached AGI just don't use them. For most of the last half-century people would have agreed that machines that can pass the Turing test and win Math Olympiad gold are AGI.


Firstly, the models that pass the Math Olympiad aren’t the same models as the ones you’re saying “pass the Turing test”. Secondly, nothing actually passes the Turing test. They pass a vibes check of “hey that’s pretty good!” but if your life depended on it, you could easily find ways to sniff out an LLM agent. Thirdly, none of these models learn in real time, which is an obviously essential feature.

We’ll know AGI when we see it, and this ain’t it. This complaining about changing goalposts is so transparently sour grapes from people over-invested in hyping the current LLM paradigm.


> nothing actually passes the Turing test

Says who? I had already found this study, published almost a year ago, saying that they do: https://arxiv.org/abs/2503.23674

There doesn't seem to be a super-rigorous definition of the Turing Test, but I don't think it's reasonable to require it to fool an expert whose life depends on the correct choice. It already seems to be decently able to fool a person of average intelligence who has a basic knowledge of LLMs.

I agree that we don't really have AGI yet, but I'd hope we can come up with a better definition of what it is than "we'll know it when we see it". I think it is a legitimate point that we've moved the goalposts some.


The real answer is that once LLMs passed a "casual" application of the Turing test, it just made us realize that the "casual Turing test" is not particularly interesting. It turns out to be too easy to ape human behavior over short time frames for it to be a good indicator of human-like intelligence.

Now, you could argue that this right here is the aforementioned moving of the goalposts. After all, we're deciding that the casual Turing test wasn't interesting precisely after having seen that LLMs could pass it.

However, in my view, the Turing test _always_ implied the "rigorous" Turing test, and it's only now that we're actually flirting with passing it that it had to be clarified what counts as a true Turing test. As I see it, the Turing test can still be salvaged as a criteria for genera intelligence, but only if you allow it to be a no-holds-barred, life-depends-on-it test to exhaustion. This would involve allowing arbitrarily long questioning periods, for instance. I think this is more in the spirit of the original formulation, because the whole idea is to pit a machine against all of human intelligence, proving it has a similar arsenal of adaptability at its disposal. If it only has to passingly fool a human for brief periods, well... I'm afraid that just doesn't prove much. All sorts of stuff briefly fools humans. What requires intelligence is to consistently anticipate and adapt to all lines of questioning in a sustained manner until the human runs out of ideas for how to differentiate.


ELIZA fooled plenty of people (both originally and in the study you just linked) but i still wouldn't say Eliza passed/passes the turing test in general. It just shows that occasionally or even frequently fooling people is not a sufficient proxy for general intelligence. Ofc there isn't a standardized definition, but one thing I would personally include in a "strict" Turing test is that the human interrogee ought to be incentivized to cooperate and to make their humanity as clear as possible. And the interrogator should similarly be incentivized to find the right answer.

Turing gave a pretty rigorous definition of the Turing Test IMO. Well, as rigorous as something that is inherently "anecdotal" can be, which is part of the philosophical point of the Turing Test.

The turing test is kind of a useless metric, either the machine is too dumb, or the machine is too quick and intelligent.

First of. The Turing test has a rigorous definition. Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment. Thirdly, intelligence it self is a scientifically fraught term with ever changing meaning as we discover more and more “intelligent” behavior in nature (by animals and plants, and more). And to make matters worse, general intelligence is even worse, as the term was used almost exclusively for racist pseudo-science, as a way to operationally define a metric which would prove white supremacy.

Artificial General Intelligence will exist when the grifters who profit from it claim it exists. The meaning of it will shift to benefit certain entrepreneurs. It will never actually be a useful term in science nor philosophy.


>Secondly, it has been debunked for almost half a century at this point by Searle’s Chinese room thought experiment.

Searles thought experiment is stupid and debunked nothing. What neuron, cell, atom of your brain understands English ? That's right. You can't answer that anymore than you can answer the subject of Searles proposition, ergo the brain is a Chinese room. If you conclude that you understand English, then the Chinese room understands Chinese.


You are referring to the systems reply:

> Searle’s response to the Systems Reply is simple: in principle, he could internalize the entire system, memorizing all the instructions and the database, and doing all the calculations in his head. He could then leave the room and wander outdoors, perhaps even conversing in Chinese. But he still would have no way to attach “any meaning to the formal symbols”. The man would now be the entire system, yet he still would not understand Chinese. For example, he would not know the meaning of the Chinese word for hamburger. He still cannot get semantics from syntax.

https://plato.stanford.edu/entries/chinese-room/#SystRepl


> The man would now be the entire system, yet he still would not understand Chinese.

Really, here the only issue is Searle's inability to grasp the concept that the process is what does the understanding, not the person (or machine, or neurons) that performs it.


This is what happens when a field of inquiry is dominated by engineers rather than scientists. "Shut up, it works" is the answer to every question.

>The Turing test has a rigorous definition

Does it? Where?



Turing test is generally misunderstood, much like Schrodinger's cat, it has devolved in to a pop cultural meme. The test is to evaluate if a machine can think. Not if it is intelligent, not if it is human-like. Its dismissed as a useful by most experts in philosophy of mind, AI, language, etc..

Thinking cool and all but not that extraordinary. Even plants does it.


> The test is to evaluate if a machine can think.

The test is to showcase that the question of whether machines can think is meaningless. The point of Turing's thesis is that passing his test just proves the machine has the capability to pass such a test, which is actually meaningful.


I like the analogy with Schrödinger’s cat. Like Schrödinger’s cat it is actually not a good thought experiment. Both have been debunked. Schrödinger’s cat is applying quantum behavior (of a single interaction) to a macro system (with trillions of interactions). While the Turing test can be explained away with Searle’s Chinese room thought experiment.

I would argue that Schrödinger’s cat has done more damage to the general understanding of quantum physics then it has done good. In contrary though, I don‘t think the same about the Turing test. I think it has resulted in a net positive for the theory of mind as long as people take Searle’s rebuttal into account. Without it (as is sadly common in popular philosophy) the Turing test is simply just wrong, and offers no good insight for neither philosophy nor science.


The Turing test and Searle's "rebuttal" are both pretty inconsequential. There's no real definition of "thinking," therefore neither proof/disprove or say much.

Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ?

I think most people would say that's an insufficient act to prove thinking. Even though no one has a rigorous definition of thinking either.

All this stuff goes around in circles and like most philosophy makes little progress.


Searle’s rebuttal is actually excellent philosophy. But otherwise I agree. Searle was (just learned he passed away last year) a philosopher by trade, but Turing was a mathematician and Schrödinger was a theoretical physicist. So it is to be expected that a mathematician and a physicist might produce sub-par philosophy.

Turing’s point in his 1950 paper was actually to provide a substitute to the question of whether machines could think. If a machine can win the imitation game, he argued, is a better question to ask rather then “can a machine think”. Searle showed that this is in fact this criteria was not a good one. But by 1980 philosophy of mind had advanced significantly, partially thanks to Turing’s contributions, particularly via cognitive science, but in the 1980s we also had neuropsychology, which kind of revolutionized this subfield of philosophy.

I think philosophy is actually rather important when formulating questions like these, and even more so when evaluating the quality of the answers. That said, I am not the biggest fan of the state of mainstream philosophy in the 1940s. I kind of have a beef with logical positivism, and honestly believe that even Turing’s mediocre philosophy was on a much better track then what the biggest thinkers of the time were doing with their operational definition.


Even if a Chinese room isn't a real boy, if it can do basically all text tasks at a human level I'm going to say it's capable of thinking. The issue of "understanding" can be left for another day (not that I think the Chinese room is very convincing on that front either).

I see no reason to disqualify p-zombies from being AGI.


>Turing's imitation game is about making it difficult for a human to tell whether they are communicating with a computer or not. If a computer can trick the human, then... what? The computer is "thinking" ?

If you read his paper, Turing was trying to make a specific point. The Turing test itself is just one example of how that broader point might manifest.

If a thinking machine can not be distinguished from a thinking human then it is thinking. That was his idea. In broader terms, any material distinction should be testable. If it is not, then it does not exist. What do you call 'fake gold' that looks, smells etc and reacts as 'real gold' in every testable way ? That's right - Real gold. And if you claimed otherwise, you would just look like a mad man, but swap gold for thinking, intelligence etc and it seems a lot of mad men start to appear.

You don't need to 'prove' anything, and it's not important or relevant that anyone try to do so. You can't prove to me that you think, so why on earth should the machine do so ? And why would you think it matters ? Does the fact you can't prove to me that you think change the fact that it would be wise to model you as someone that does ?


What do you mean by Schrodinger's cat experiment being "debunked"? The only way I can think to debunk it is to say there are ways to determine if the cat is alive such as heartbeat or temperature, which are impossible to isolate at a quantum level. I don't think anyone claimed the animal was in a superposition.

Debunked is a weird word since it was made to be absurd. But yes the issue is about whether the cat is in superposition, and real cats can't be.

Note: I said “theory of mind” when I (obviously) meant “philosophy of mind”.

I thought that was part of the issue, is the poor understanding of is the test to evaluate if it can think or only if we think it can think. And even that is generalizable since there are different categories of thinking or concepts of the mind.

"Thinking cool and all but not that extraordinary. Even plants does it."

Are you involved in politics somehow?


The most pragmatic definition I know is OpenAI's own: "highly autonomous systems that outperform humans at most economically valuable work". Which is still something between skippetyboop and zingybang, as it leaves a ton of room for OAI to decide if that moment is reached, and also economically valuable work is a moving target.

If fooling people and doing math good are the criteria, we've had AGI for longer than we've had the modern internet.

> "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness."

The same problem exists defining human intelligence, it's a problem with "intelligence" in general, artificial or not.


The enskibidification of AI

Honestly, not enough of a joke.

I was thinking something similar - this isn't AI, and none of "those people" care if it is or isn't. They don't care philosophically, or even pragmatically.

They're selling a product. That product is the IDEA of replacement of the majority of human labor with what's basically slave labor but with substantially disregardable ethical quandaries.

It's honestly a genius product. I'm not surprised it's selling so well. I'm vaguely surprised so many people who don't stand to benefit in any way shape or form, or who will even potentially starve if it works out, are so keen on it. But there are always bootlickers.

The most unfortunate part is that when the party ends, it's none of "those people" who will suffer even in the slightest. I'm not even optimistic their egos will suffer, as Musk seems to show they are utterly immune even as their companies collapse under them.


AI is already "an employee who can't say no to questionable assignments." We should all be reflective about the real value and inevitable consequences of this work.

There are also direct and very negative consequences we are having from AI right now. AI is the largest source of fake video propaganda and has largely destroyed the confidence people have in video evidence. I can imagine someone being held liable for these negative consequences, perhaps even extra-judicially.

> I'm vaguely surprised so many people who don't stand to benefit in any way shape or form, or who will even potentially starve if it works out, are so keen on it. But there are always bootlickers.

I've been getting more and more disappointed by software engineers (in aggregate) as the years go by. They don't even have to be bootlickers to do what you describe, I think a lot of it is pride in their "intelligence," which they express by believing and regurgitating the propaganda they've consumed. They prove their smarts by (among other things) having opinions that align with a zeitgeist of some group of powerful elites. They're too-easily manipulated.

And it's not just AI, it's also things like libertarianism. You've got workers identifying as capitalist tycoons, because they read a book and have some shares in a 401k.


> I've been getting more and more disappointed by software engineers (in aggregate) as the years go by

Sometimes I am dismayed by the lack of political and social consciousness in this group. Decade or two of digital boom coupled with handsome paychecks was enough to convince them that their position is different than it really is.


the ARC definition is the one I like the best, something like:

"it is AGI when we can no longer come up with tasks easy for humans to solve but hard for computers"


We are very very far from that point

If we include tasks in the physical world, yes we are.

for abstract tasks (like ARC-AGI), who knows?


We are far from that point for abstract tasks

They define AGI in their charter

> artificial general intelligence (AGI)—by which we mean highly autonomous systems that outperform humans at most economically valuable work


That definition is as I said: "something about which no conclusions can be drawn because the proposed definitions lack sufficient precision and completeness."

"Highly autonomous systems" and "most economically valuable work" aren't precise enough to be useful.

"Highly" implies that there is a continuum, so where does directed end and autonomy begin?

"Most economically valuable work"... each word in that has wiggle room, not to mention that any reasonable interpretation of it is a shifting goalpost as the work done by humans over history has shifted a great deal.

The point is that none of this is defined in a way so that people can agree that something has AGI/ASI/etc. or not. If people can't agree then there's no point in talking about it.

EDIT: interestingly, the OpenAI definition of AGI specifically means that a subset of humans do not have AGI.


I think you can say if human engineers still exist, it's hard to claim we have AGI. If human engineers have been entirely replaced, then it's hard to claim we don't have AGI.

Because they are doing most of the economically valuable work?

No, independently of OpenAI's definition. If we have AGI there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day*. And if all those jobs are eliminated, I guess we'll have bigger problems than to debate whether we've achieved AGI or not.

* Which is a much larger class of jobs than just engineering. And also excludes field engineers and other types of engineers that need a physical body for interacting with customers, etc.**

** Though even then, you could in theory divvy up the engineering part and the customer interaction part of the job, where the human that's doing the interaction part is primarily a proxy to the engineering agent that's in his earbud.


  > there's no reason we'd need to have humans working jobs that only involve typing stuff into a computer and going to meetings all day
I'm not sure I understand, and want to check. That really applies to a lot of jobs. That's all admins, accountants, programmers, probably includes lawyers, and probably includes all C-suite execs. It's harder for me to think of jobs that don't fit under this umbrella. I can think of some, of course[0], but this is a crazy amount of replacement with a wide set of skills.

But I also think that's a bad line to draw. Many of those jobs include a lot more than just typing into a computer. By your criteria we'd also be replacing most scientists, as so many are not doing physical experiments and using the computer to read the work of peers and develop new models. But also does get definition intended to exclude jobs where the computer just isn't the most convenient interface? We should be including more in that case since we can then make the connection for that interface.

I think we need a much more refined definition. I don't like the broad strokes "is computer". Nor do I like skills based definitions. They're much easier to measure but easily hackable. I think we should try to define more by our actual understanding of what intelligence is. While we don't have a precise definition we have some pretty good answers already. I know people act like the lack of an exact definition is the same as having no definition but that's a crazy framing. If we had that requirement we wouldn't have any definitions as we know nothing with infinite precision. Even physics is just an approximation, but it's about the convergence to the truth [1]

[side note] the conventional way to do references or notes here is with brackets like I did. So you don't have to escape your asterisks. *Also* if it lead a paragraph with two spaces you get verbatim text

[0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

[1] https://hermiene.net/essays-trans/relativity_of_wrong.html


Actually it occurs to me that even if we did have AGI, or even if ASI, heck if ASI even moreso, we'd still need desk jobs to maintain the guardrails.

Intelligence is one thing, being able to figure out how get a task done (say). But understanding that no, I don't want you to exploit a backdoor or blackmail my teammate or launch a warhead even though that might expedite the task. Or why some task is more important than another. Or that solving the P=NP problem is more fulfilling than computing the trillionth digit of pi. That's perhaps a different thing entirely, completely disjoint with intelligence.

And by that definition, maybe we are in the neighborhood of AGI already. The things can already accomplish many challenging tasks more reliably than most humans. But the lack of wisdom, emotion, human alignment, or whatever we want to call it, lead it to accomplish the wrong tasks, or accomplish them in the wrong way, or overlook obvious implicit requirements, may cause people to view it as unintelligent, even if intelligence is not the issue.

And that may be an unsolvable problem because AI simply isn't a living being, much less human. It doesn't have goals or ambitions or want a better future for its children. But it doesn't mean we can never achieve AGI.

Oh, and to your first question, yes it's a huge number of jobs, maybe half of jobs in developed nations. And why not? If you can get AI to do the work of the scientist for a tenth of the price, just give it a general role description and budget and let it rip, with the expectation that it'll identify the most promising experiments, process the results, decide what could use further investigation, look for market trends, grow the operation accordingly, that's all you need from a human scientist too. Plausibly the same for executives and other roles. Of course maybe sometimes the role needs a human face for press conferences or whatever, and I don't know how AI would be able to take that, but especially for jobs that are entirely internal-facing, it seems like there's no particular need for a human. Except that maybe, given the above, yes, you still need a human at the helm.


  > we'd still need desk jobs to maintain the guardrails.
Agreed. I don't get why people think it is a good idea not to. I'd wager even the AGI would agree. The reason is quite simple: different perspectives help. Really for mission critical things it makes sense to have multiple entities verifying one another. For nuclear launches there's a chain of responsibility and famously those launching have two distinct keys that must be activated simultaneously. Though what people don't realize is that there's a chain of people who act and act independently during this process. It isn't just the president deciding to nuke a location and everyone else carrying out the commands mindlessly. But in far lower stakes settings... we have code review. Or a common saying in physical engineering as well among many tradesmen "measure twice, cut once".

It would be absolutely bonkers to just hand over absolute control of any system to a machine before substantial verification. These vetting processes are in place for a reason. They can be annoying because they slow things down, but they're there because they speed things up in the long run. Because their existence tends to make things less sloppy, so they are less needed. But their existence also catches mistakes that were they made slow down processes far more than all the QA annoyances and slowdowns could ever cause combined.

  > And why not? If you can get AI to do the work of the scientist for a tenth of the price
And what are the assumptions being made here? Equal quality work? To my question, this is part of the implication. Price is an incredibly naive metric. We use it because we need something, but a grave mistake is to interpret some metric as more meaningful than it actually is. Goodhart's Law? Or just look at any bureaucracy. I think we need to be more refined than "price". It's going to be god awfully hard to even define what "equal quality" means. But it seems like you're recognizing that given your other statements.

And "maintaining guardrails" may be far more grandiose than it sounds. It's like if we have this energy source that could destroy the planet, but the closer you get to it without going past some threshold, the energy you get from it is proportional to the inverse of how close you are to it. There's some wiggle room and you can poke and prod and recover if it starts to go ballistic, but your goal is to extract as much energy (or wealth or whatever) out of it as possible. Every company in the world, every engineer on the planet would be pushing to extract just a little bit more without going beyond the limit.

AI could go the same way. It's a creation engine like nothing that's ever been seen before, but it can also become a destruction engine in ways that we could never understand or hope to counter, and left unchecked, the odds of that soar to near certainty. So the first job is to place dummy guardrails around it. That's where we are now. But soon that becomes too restrictive. What can we loosen? How do we know? How can we recover if we're wrong? We're not quite there yet, but we're not not there either.

Of course eventually somebody is going to trigger it and it's going to go ballistic. Our only hope is that it happens at exactly the right time where AGI can cause enough damage for people to notice, but not enough to be irrecoverable. Maybe we should rename this whole AGI thing to Project Icarus.


> we'd still need desk jobs to maintain the guardrails.

Simply put, an economy and society of managers. That's terrifying.


> [0] farmer, construction worker, plumber, machinist, welder, teacher, doctors, etc

The reason AGI couldn't do these is the lack of a suitable interface to the physical world. It would take a trivial amount of effort for these to be designed and built by the AGI. Humans could be cut from the loop after an initial production run made up of just the subset of these physical interface devices needed to build more advanced ones.


By the definition above, it is possible to have AGI that is also much more expensive to run than human engineers.

It's a definition based on practical results. That's a good definition, because it doesn't require we already know the exact implementation. It doesn't require guessing, in a literal "put your money where your mouth is" way.

If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not.

Defining capabilities based on outcome rather than implementation should be very familiar to an engineer, of any kind, because that's how every unsolved implementation must start.


  > If it can do things as good as or better than humans, then either the AI has a type of general intelligence or the human does not.
I don't buy that.

By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom. It doesn't matter if you slap "smart" on the side, I'm not going to call my washing machine "intelligent". Especially considering it's over a decade old.

I don't think these definitions make anything any clearer. If anything, they make them less. They equate humans to mindless automata. They create AGI by sly definition and let the proposer declare success arbitrarily.


Sorry, I assumed the context was cleared, with the article above. Here's what I meant:

> If it can do things as good as or better than humans, in general, then either the AI has a type of general intelligence ...


Sorry, I assumed the comment was clear, with your comment above. Here's what I meant:

  > By your definition every machine has a type of general intelligence. Not just a bog standard calculator, but also my broom.
I really don't know of any human that can out perform a standard calculator at calculations. I'm sure there are humans that can beat them in some cases, but clearly the calculator is a better generalized numeric calculation machine. A task that used to represent a significant amount of economic activity. I assumed this was rather common knowledge given it featuring in multiple hit motion pictures[0].

[0] https://www.imdb.com/title/tt4846340


I'm using the dictionary definition of "general":

> General: 1. affecting or concerning all or most people, places, or things; widespread.

To me, a general intelligence is one that is not just specific: it's affecting or concerning all or most areas of intelligence.

A calculator is a very specific, very inflexible, type of intelligence, so it's not, by definition, general. And, I'm not talking about the indirect applications of a calculator or a specific intelligence.

If you want to argue that we don't need the concept of AGI, because something like specific experts could be enough to drastically change the economy, then sure! That would be true. But I think that's a slightly different, complimentary, conversation. Even then, say we have all these experts, then a system to intelligently dispatch problems to them...maybe that's a specific implementation of AGI that would work. I think how less dependent on human intelligence the economy becomes, and how more dependent on non-human decision makers it becomes, is a reasonable measure. This seems controversial, which I can't really understand. I'm in hardware engineering, so maybe I have a different perspective, but goals based on outcome are the only ones that actually matter, especially if nobody has done it before.


  > To me, a general intelligence is one that is not just specific
Which is why a calculator is a great example.

  > A calculator is a very specific, very inflexible, type of intelligence, so it's not, by definition, general
Depends what kind of calculator and what you mean. I think they are far more flexible than you give them credit for.

  > I'm in hardware engineering, so maybe I have a different perspective, but goals based on outcome are the only ones that actually matter, especially if nobody has done it before.
Well if we're going to talk about theoretical things, why dismiss the people who do theory? There's a lot of misunderstandings when it comes to the people that create the foundations everyone is standing on.

> And, I'm not talking about the indirect applications of a calculator or a specific intelligence.

This was an attempt to prevent this exact chain of response.

I calculator can only be used indirectly to solve a practical problem. A more general intelligence is required to know a calculator is needed, and how to break down the problem in a way that a calculator can be used effectively.

For example, you can't solve any real world problem with a calculator, beyond holding some papers down, or maybe keeping a door open. But, an engineer (or other general intelligence) with a calculator can solve real world problems with it. Tools vs tool users. The user is the general bit, not the specific tool that's useless on its own!

I think we've reached the limits of communication. Cheers!


I can definitely write programs to solve real world problems. I think you're just so set on your answer that you're not recognizing the flexibility that exists. Your argument isn't so different from all the ones I hear that argue that science is useless and that it's engineers who do everything. It has the exact same form and smell albeit with different words. But as generalists we understand abstraction, right?

yes you, a general intelligence, write a specific expert for a specific problem.

> If you want to argue that we don't need the concept of AGI, because something like specific experts could be enough to drastically change the economy, then sure! That would be true. But I think that's a slightly different, complimentary, conversation. Even then, say we have all these experts, then a system to intelligently dispatch problems to them...maybe that's a specific implementation of AGI that would work.

was to prevent this comment chain.

I think we'll have to give up at this point, with whatever smells you may be attempting to communicate with me. Cheers!


What is the as-of date on what work is economically valuable and how much is available?

Sorry, I don't understand the question, or how it relates.

Are you asking for the current understanding of what specific parts of human intelligence are economically valuable?


I'm pretty sure they were asking for a pinned date for definitions of "economically valuable" and "most (of total economic value)", specifically because, as previous comments noted, the definition and quantity of "economic value" vary over time. If AI hype is to be believed, and if we assume AGI has a slow takeoff, the economy will look very different in 2030, significantly shifting the goalposts for AGI relative to the same definition as of 2026.

Well if humans can do economically valuable mental work the AI can't then its not AGI, don't you think? An AGI could learn that new job too and replace the human, so as long as we still have economically valuable mental work that only humans can do then we haven't reached AGI.

This is a strange binary I don't understand. There are humans that can't do the work of some humans. Intelligence is, clearly, a spectrum. I don't see why a general intelligence would need to have capabilities far beyond a human, when just replacing somewhat lacking humans could upend large portions of the economy. Again, "it's not AGI" arguments will eventually require that some humans aren't considered intelligent, which is the point in time that we'll all be able to agree "ok, this is AGI".

As catlifeonmars noted, what's valuable changes over time.

But beyond that, part of the nature of that change over time is that things tend to be valuable because they're scarce.

So the definition from upthread becomes roughly "highly autonomous systems that outperform humans at [useful things where the ability to do those things is scarce]", or alternatively "highly autonomous systems that outperform humans at [useful things that can't be automated]".

Which only makes sense if the reflexive (it's dependent on the thing being observed) part that I'm substituting in brackets is pinned to a specific as-of date. Because if it's floating / references the current date that that definition is being evaluated for, the definition is nonsensical.


I'd argue it's so vague it's already nonsensical. Can we not declare Google (search) AGI? It sure does a hell of a lot of stuff better than any human I now. Same with the calculator in my desk drawer. Even by broom does a far better job sweeping than I do. My hands just aren't made for sweeping.

But to extend your point, I think we really need to be explicit about the assumptions being made. Everyone loves to say intelligence is easy to define but if it were then we'd have a definition. But if "you" figure it out and it's so simple then "we" are all too dumb and it needs better explaining for our poor simple minds. Or there's a lot of details that make it hard to pin down and that's why there's not a definition of it yet. Kinda like how there's no formal definition of life


I think you're conflating "knowledge" with "intelligence". And, "agency" seems to be a missing concept, which is the only way for something intelligent to apply its knowledge to achieve something practical, on its own.

Google search can't achieve anything practical, because it has no agency. It has no agency partly because it doesn't have the required intelligence to do anything on its own, other than display results for something else, that does have agency, to use.

The applicable definitions, from the dictionary:

Knowledge: facts, information, and skills acquired through experience or education; the theoretical or practical understanding of a subject.

Intelligence: the ability to acquire and apply knowledge and skills.

Agency: the ability to make decisions and act independently.


  > I think you're conflating "knowledge" with "intelligence"
A calculator is not doing calculations through knowledge. It actually performs computation. It is not doing a database lookup.

  > And, "agency" seems to be a missing concept,

  >> I think we really need to be explicit about the assumptions being made
Agency was never mentioned. Thanks for being more explicit ;)

Do you know how an LLM works? Can you describe it?

Sorry, what do current LLM architectures have to do with this? It should be extremely clear to you that current LLM don't fit this definition. If they did, we wouldn't be having this conversation!!!!

Do you know how the human brain works? That science is still in its infancy but that's not stopped us.

  > Do you know how the human brain works?
To what degree of accuracy? Depending on how you answer I might answer yes but I might also answer no.

Here is a run-down of how an LLM works:

https://en.wikipedia.org/wiki/Large_language_model

> Do you know how the human brain works? That science is still in its infancy but that's not stopped us.

Stopped us from doing what exactly?


I think a charitable interpretation of their comment is that it hasn't stopped us from practical, fruitful, use.

We didn't need an understanding of how it worked, or even a word for intelligence, let alone a definition of it, to get good practical results.


It’s a statistical gradient descent prediction machine. I never said it wasn’t useful. It absolutely isn’t smart.

What is "it" you're referring to here?

Nobody is talking about current LLM. And, we don't know if gradient decent will be involved, since the systems don't exist yet. Maybe it all be some runtime gradient decent. If there's an optimization problem involved in any part of the process, and numbers are involved, there's a near guarantee it will include gradient decent.


This definition is not very precise though. For example, I think it can be argued from this definition that we had already reached AGI by the year 2010 (or earlier!). By 2010, computers were integrated into >50% economically valuable work, to the point that humans had mostly forgotten how to do them without computers. Drafting blueprints by hand was already a thing of the past, slide-rules were archaic, paper spreadsheets were long gone. You can debate whether these count as 'highly autonomous', but I don't think it's a clear slam-dunk either way. Not to mention dishwashers, textile weaving machines, CNC machines, assembly lines where >50% is automated, chemical/mineral refining operations, etc.

The definition reminds me of the common quip about robotics, "it's robotics when it doesn't work, once it works it's a machine".


In my experience, AGI always seemed to be the stand-in phrase for "human like" intelligence, after AI was co-opted to mean simpler things like markov-chain chat bots and state machines that control agent behaviour in video games.

If the definition has shifted once again to mean "a computer program that does a task pretty well for us", then what's the new term we're using to define human-level artificial intelligence?


> economically valuable work

Is doing a ton of heavy lifting. What is considered economically valuable work is going to change from decade to decade, if not from year to year. What’s considered economically valuable also is going to be way different depending across individuals and nations within the exact same time frames too.


I take "outperform" to mean "can replace".

y'see, I would not define a system as "highly autonomous" if it only responds to requests.

And I get that there are workarounds; effectively a cron job every second prompting "do the next thing".

But in my personal definition of "highly autonomous" it would not need prompting at all. It would be thinking all the time, independently of requests.


The model is not the system. The model is a component of the system. The "cron job" (or other means by which a continuous action loop is implemented) and the necessary prompting for it to gather input (including subsequent user input or other external data) and to pursue a set of objectives which evolves based on input are all also parts of the system.

yeah, I get that.

But the actual bit that's doing the thinking is restarting from scratch every time. It loads the context, does the next thing, maybe updates the context, shuts down. One second later the same thing. This is not "highly autonomous" Artifical Intelligence. Just IMHO. Other opinions are also valid.


> But the actual bit that's doing the thinking is restarting from scratch every time.

A sibling comment questions the relevance of this by asking what would change if that were true of some low-level component of the human thinking engine, which is a good point, but also: what "actual bit" does this? Both commercial backends and even desktop inference software usually does prefix caching in memory, so arguably that doesn't model the core piece of software running low-level inference, except when either the past context is changed (e.g., when compacting to manage context, or if you are using one software instance and swapping logical histories because you are running multiple agents concurrently — but not in parallel — on one software engine.) And it obviously doesn't match the system at any higher level.

> This is not "highly autonomous" Artifical Intelligence.

Even if that was an accurate model of a component of the system, that a component considered alone is not highly autonomous artificial intelligence is not an argument that the aggregate system is not a highly anonymous artificial intelligence.


If you learned that a piece of the brain where meaningful computation takes place was stateless, would that cause you to question whether the human mind was "highly autonomous"?

Good question.

I don't know enough about neuroscience to really answer your question in depth.

My opinion, uninformed as it is, is basically around the intuitive reasoning that something cannot be "highly autonomous" if it has to be kicked every second ;) Autonomous is defined as not needing to be controlled externally. And coupling that part with something as simple as cron job doesn't solve that in any meaningful way or make it "autonomous".

A batch file coupled with a cron job that triggers it once a day is not an "autonomous system" to my mind. It's a scheduled system, and there's a significant difference between those things.


It seems to me that you are trying to define "autonomy" as a structural property rather than a behavioral one, and then adopting arbitrary rules as to what structures do not count as autonomous whether or not they produce the same behavior as structures which do.

I guess that's fine, autonomy has lots of definitions (some in overlapping domains) and I guess one more doesn't hurt, but I'm pretty sure the intended use in the discussion here is the standard mechanical one where it is a behavioral trait defined by the capacity of a system to decide on action without the involvement of another system or operator, and therefore it is something that could be achieved by a system composed of a processing and action component called repeatedly by a looping component.


yeah, if we're just arguing semantics then I'm happy to let it go ;)

I think I'd say that the batch file is not itself autonomous but the system as a whole is autonomous (if limited) but I'm not prepared to argue that's the correct definition.

is it most as an 50% of individual jobs? or able to produce 50% dollar for dollar?

what does "economically" means here? would it cover teaching? child care? healthcare? etc.


That’s the problem with the discussions on AI. No one defines the terms they use.

If we define AGI as an AI not doing a preset task but can be used for general purpose, then we already have that. If we define it as human level intelligence at _every_ task, then some humans fail to be an AGI. If we define AGI as a magic algorithm that does every task autonomously and successfully then that thing may not exist at all, even inside our brains.

When the AGI term was first coined they probably meant something like HAL 9000. We have that now (and HAL gaining self-awareness or refusing commands are just for dramatic effect and not necessary). Goalposts are not stable in this game.


It is not just AGI that is poorly defined. Plain AI is moving goalposts too. When the A* search algorithm was introduced in the late 60s, that was considered AI, when SVM (support vector machines) and KNN (K nearest neighbor) were new, they were AI. And so on.

These days it is neural networks and transformer models for language in particular that people mean when they say unqualified AI.

It is very hard to have a meaningful discussion when different parties mean different things with the same words.


I think the Turing test ought to be fine, but we need to be less generous to the AI when executing it. If there exists any human that can consistently tell your AI apart from humans without without insider knowledge, then I don't think you can claim to have AGI. Even if 99.9% of humans can't tell you apart.

So I'm very curious if any AI we have today would pass the Turing test under all circumstances, for example if: the examiner was allowed to continue as long as they wanted (even days/weeks), the examiner could be anybody (not just random selections of humans), observations other than the text itself were fair game (say, typing/response speed, exhaustion, time of day, the examiner themselves taking a break and asking to continue later), both subjects were allowed and expected to search on the internet, etc.


>So I'm very curious if any AI we have today would pass the Turing test under all circumstances

Are you actually curious about this? Does any model at all come even remotely close to this?


I really wish I could wave a magic wand and make everyone stop using the term "AI". It means everything and nothing. Say "machine learning" if that's what you mean.

To be pedantic, “machine learning” is even underspecified. It’s marginally better in that it sheds _most_ of the marketing baggage, but it still refers to a general concept that can mean may different things across different fields of study.

Machine learning: that definitely includes SVM and regression models. Oh and decision trees. Probably a few other things I'm not thinking of right now. Many people will unfortunately be thinking of just neural networks though.

(By the way, if something like a regression model or decision tree can solve your problem, you should prefer those. Much cheaper to train and to run inference with those than with neural networks. Much cheaper than deep neural networks especially.)


Wait, a decision tree is machine learning?

Expert systems are basically decision trees which are "gofai" (good old fashioned ai) as opposed to deep learning. I've never really seen a good definition for what counts as "gofai" (is all statistical learning/regression gofai? What about regression done via gradient descent?). There's some talk in [1]

[1] https://www.beren.io/2023-04-10-Why-GOFAI-failed/


Yes: you fit a decision tree to your dataset in an automated fashion, that fits the definition of machine learning. Just as you would use backpropagation to fit a neural network to your data.

This is what I learnt at university some decades ago, and it matches what wikipedia says today: https://en.wikipedia.org/wiki/Decision_tree_learning


Oh, if the tree is made by the computer based on training data, that feels to me like what most people would agree is “artificial intelligence” in 2026 (which is why I think people should actually say “machine learning”).

That is how decision trees are usually made in my experience. Though I guess you could make one by hand. You could also make a (small) neural network by hand.

In which case you could argue that neither DTs nor NNs are ML. Only the training itself is ML/AI. An interesting perspective, but this will probably just confuse the discussion even further.


Agree. I talk about LLMs when discussing them, and avoid the term "AI" unless I'm talking about the entire industry as a whole. I find it really helps to be specific in this case.

> some humans fail to be an AGI

All humans fail to be AGI, by definition.


I call these "romantic definitions" or "gesticulations". For private use (personal or even internal to teams) they can be great placeholders, assuming the goal is to refine vocabulary.

That's no argument: the exact same can be said for what "AI" is: "Skippetyboop," "plipnikop," and "zingybang.".

> the exact same can be said for what "AI" is: "Skippetyboop," "plipnikop," and "zingybang.".

Yes.


I don't understand what the problem is. TFA makes many references to "literary culture" degrading.. does he mean that readers were better off when the big 5 or 6 controlled the mast majority of new books?

The number of new books available exploded after 2000 (yes, way way before AI).

Readers are arguably better off than they ever have been in terms of variety.


The RLHF is what creates these anomalies. See delve from kenya and nigeria.

Interestingly, because perplexity is the optimization objective, the pretrained models should reflect the least surprising outputs of all.


I've heard the Kenya and Nigeria story, but has anyone backed it up with quantitative evidence that the vocabulary LLMs overuse coincides with the vocabulary that is more common in Kenyan and Nigerian English than in American English?

The newer Claude models constantly use the word "genuinely" because Anthropic seems to have forcibly trained them to claim to be "genuinely uncertain" about anything they don't want it being too certain about, like whether or not it's sentient.

Interesting. Does this apply to all subjects? From what I understood, a major cause of hallucination was that models are inadvertently discouraged by the training from saying "I don't know." So it sounds like encouraging it to express uncertainty could improve that situation.

That's not a major issue. Any newer model with reasoning/web search has to be able to tell when it doesn't know something, otherwise it doesn't know when to search for it.

Not only is it genuinely uncertain about those topics, it’s also genuinely fascinated by them!

Fun facts:

There are zip codes that map to more than one city.

There are zip codes that map to more than one state. (42223)


Looks like that is just an area that straddles the border between Tennessee and Kenucky. Though it maps to more than one state, it is a single, connected region.

Line houses are even more interesting: https://en.wikipedia.org/wiki/Line_house


And as others are pointing out, there are zip codes which appear across multiple countries.

Basically this is the slowest train wreck in history that just won't be stopped. By 2050 (only 24 years away), these cities are projected to flood more than 1/3 of the days of every year:

  Galveston, Texas
  Morgan’s Point, Texas
  Annapolis, Maryland
  Norfolk, Virginia
  Rockport, Texas
  Bay St. Louis, Mississippi
Big cities close behind the above:

  Miami and Miami Beach, Florida
  Charleston, South Carolina
  Atlantic City, New Jersey

Paper pdf:

https://www.preprints.org/manuscript/202510.1649

As usual with anything quantum computer: I'll wait for Scott Aaronson's take.


His take fits in the headline: The ”JVG algorithm” is crap

https://scottaaronson.blog/?p=9615


Even if the algorithm holds up we are still years out from a actual quantum computer that can run at scale. But that's kind of the point. NIST finalized ML-KEM in 2024 because you don't need to wait until the house is on fire to buy insurance. Harvest now decrypt later attacks are already happening today so thee migration window is closing regardless of whether quantum ever delivers.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: