Hacker Newsnew | past | comments | ask | show | jobs | submit | hliyan's commentslogin

tech stacks -> frameworks

You've actually been primarily training a physics model, with an LLM attached to it.

Good point, and I'm actually not sure that there is a clear dividing line. I expect that once we achieve capable world models and are able to analyze their internals, we'll find that the prediction mechanisms for purely physical and for verbal/behavioral responses to the agent's actions are at least partially colocated.

As particular motivation for my intuition, I expect that we had evolutionary pressure to adapt our defense mechanisms of predicting the movements of predators and prey, to handle human opponents.


The specific counterintuitive result is mentioned toward the end of the article, and I'm having some trouble understanding it:

> when analyzing average trends in groups of children, slower reaction times to the “Go” signal were linked to increased activity in many brain regions, including the default mode network

> However, when an individual had a slower reaction time to the “Go” signal, activity decreased in the default mode network — the opposite of the group-level pattern.


Now, I don't know anything about neuroscience or brain development, but hopefully I can explain the statistics in a way useful to you.

Imagine there are two groups A and B. One group, A, has slower reactions on average and high average activity The other, Group B, has higher reactions and lower than the Group A's activity. Yet inside both groups the general trend is that if someone is slower than the average reaction of their group then they're also below the average activity for their group.

If we look at the overall means without distinguishing groups, slower reaction is correlated positively with higher activity (kids from group A have higher activity and slower reaction in general, which pushes the correlation upwards. As long as the relationship in Group B isn't too strong the upward trend from Group A can easily dominate overall correlation) but inside each group the trend is actually the opposite.

This applies pretty much every time you're comparing samples. If I understood your quote correctly, they're studying a child's reaction time vs activity level by comparing the same kid in different times. The same logic applies, a person can exhibit the opposite trend to the populational average due to the same mechanism above. This can be even more dramatic, because once you start looking at averages you start losing time dependency information.

More broadly (and more formally), multivariate covariance splits in within-group and between-group terms, so if the signs of the terms are different the magnitude of one can dominate the overall sum and flip the sign.


This is a very good explanation of Simpson's paradox, which is the name for this thing.

It can go arbitrarily deep and the trend can flip sign for each added controlled variable.


Hmm I think all these replies are overcomplicating things.

At a group level, some kids are slower at this Stop/Go task than others. The group difference appears to be this increased broad-scale brain activity: the slow group is overall more prone to distraction and daydreaming.

However, at an individual level, slowing down on the task means increasing your focus (and decreasing brain activity in irrelevant regions), regardless of whether you were in the slow group or the fast group. So the group-level difference is not necessarily as profound as it might appear, and applying "slow group" with too broad a brush means you're going to sweep up some kids who are naturally cautious and focused.


Acting on your first impulse is fast (default mode).

Denying that first impulse, thinking about it, and then acting is slow.


One way to think of it — I didn't read the article in depth so this is just an example — is in terms of overall individual differences in speed and activity level. Then, you could have slower persons having increased activity relative to faster persons, but it still be true that when a slower person had an even slower signal reaction, their activity went down, and when a faster person had a slower signal reaction, their activity went down as well.

It's a classic psychological phenomenon, where individual differences are obscuring time course patterns and vice versa.

Of course, this sidesteps the question of why (in the hypothetical example) the overall individual differences exist. Assuming those general individual differences are reliable and "real", you still have to explain why they are there, and if they predict significant outcomes, why they do, and so forth.

The message of the paper is good, although I think the press release (not surprisingly) overstates the significance of the paper. I think these kinds of issues have received a lot more attention in the literature in the last decade or so in neuroscience. It also sort of sidesteps a lot of the more thorny questions about truly person-specific patterns and how to determine when they're meaningful.


I think the plot here explains it well

https://en.wikipedia.org/wiki/Simpson%27s_paradox


The answer may be agentic loops that keeps cycling through the same problem again and again until they land on a non-erroneous outcome. Some people boast having multiple such agents working in parallel on different problems, tending to one while another is processing, perhaps not unlike the movie mad scientist who runs around the lab throwing switches while laughing maniacally at the prospect of his impending success.

Do typesetters inexplicably change the meaning of the book or document being typeset? Do compilers alter the behavior intended by the programmer, sometimes in ways that are not immediately obvious? Did the invention of typesetters lead to investments so massive, that the investors had to herald the end of handwriting (no equivalent analogy for compilers)?

It reminds me of the guy who replaced his static blog deployment scripts with asking chatgpt to generate the html from his text based on a template, and said that he isn't sure that the llm isn't changing his writing but hopes it isn't

On compilers, you know they do! Compilers have bugs and some languages have undefined behavior.

On typesetters and investment: the WYSIWYG word processor is on almost every home and office desk in the world.


> Look at how long Uber and Tesla have lasted

In a system of open-ended growth, yes, you can point to how long the system has persisted as evidence of its longevity. But in a system of plateauing growth, the system's age is an indicator of how close it may be to death. I suspect that the model that permitted the "success" of Uber and Tesla is nearing the end of its lifetime.


I absolutely cannot stand people who recite this quote but has no knowledge of the sentences that come before or after it: "We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%."


Don't forget:

- The customer is always right in matters of taste

- Jack of all trades, master of none, but oftentimes better than a master of one

- Curiosity killed the cat, but satisfaction brought it back

- A few bad apples spoil the barrel

- Great minds think alike, though fools seldom differ

Even "pull yourself up by your bootstraps" was originally meant to highlight the absurd futility of a situation.


Found this bit particularly interesting:

> We need at least one model from Anthropic and one from Google to use Passmark's multi-model consensus features.

https://github.com/bug0inc/passmark?#quick-start


When I read the phrase "it's cool to care", the first thing that popped into my mind was this line from the recent Superman movie: "Maybe that's the real punk rock". Like Alex's story landed for you, that phrase really landed for me, especially in the backdrop of how non-badass the Corenswet Superman was compared to the Snider character. (Spoilers follow) He allows himself to care enough to a point that would make him vulnerable: he gets flustered when aggressively questioned by Lane, he feels despondent during captivity, and he loses his temper when the dog is abducted. But the important thing is that he recovers quickly and shrugs off the experiences rather than brood on them. Perhaps the lack of that skill is what leads some people to adopt being aloof/disinterested as a defense mechanism.


Given this is failing due to HN hug of death, might I suggest that you do a periodic batch, save the results and serve static?


Thank you very much. Great call


Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: