I think this can be falsified by just considering the history of humanity. It wasn't that long ago that human language literally did not even exist. And our collective knowledge wasn't all that much more than 'poke him with the pointy end'. Somehow we went from that to putting a man on the Moon, unlocking the secrets of the atom, and more. And if you consider how awful we are at retaining/sharing information and just general inefficiencies due to the fact that we're humans and not just logical information processing machines, we did all of this in little more than the blink of an eye. This is something that seems to certainly be rather special.
All that humanity has achieved happened due to the simple loop of identifying a desire/need and finding a way to satisfy it. Also known as reinforcement learning. The only thing that really differentiates humans from machines is... history. We've been learning and passing on our knowledge to successive generations over millennia. Nothing really special there; give the machines a few years to learn and see what happens.
None, yet. But you can be 100% sure it's something we'll eventually succeed in adding, as it's through the guidance of desires and needs that intelligence really expresses.
Reinforcement learning requires a well defined goal and a well defined way to quantitatively measure progress along that goal. In reality these don't exist without a hand of God guiding you. In the case of machine learning that hand of God is our own. Even given infinite processing power, you could not construct a reinforcement learning system that would mimic humanity's progress - it simply is a nonstarter due to the nature of reinforcement learning itself.
Conceptually, it's really not as hard as you make it seem. There are layers, but once you peel them away there's only one thing left, which all living things share: the drive to survive (maintain internal state parameters within a certain range by accessing nutrition, protection from environmental elements, security from other survival-seeking entities, reproduction to pass on genes, etc). No need to bring God/gods into it.
There's also no need to specifically mimic humanity's progress; that's just an accident of survival facilitated by opposable thumbs and language ability. We've already made machines with the base abilities, and emulated the drive (see evolutionary algorithms[0] for example). We just need to put it all together in a few units and let them "loose" to evolve on their own for a while. It took humans ~300,000 years to get where we are today; I'm positive that it'll take machines a small fraction of that. Nothing special.
Very little, certainly approximately 0%, of what humanity has done has been driven by base survival instincts. You're describing the process to mimic a roach, not a human.
You do work, don't you (or at least are in school)? Do you do it for sheer fun? Or to be able to afford things that allow you to survive?
Even those who do something like art "for fun" do it because it sates an internal need (not all actions need to make sense, because the inherent randomness of evolution is messy and leaves artifacts). Though also the desire to create some form of legacy can be considered a kind of survival: to be remembered by others beyond one's own lifespan.
I’m not claiming an LLM is structurally or functionally equivalent to a human brain. I just said that what we call “creativity” is in fact a very derivative thing.
