> As in, are these two tasks (captioning an image and replying to a natural language prompt) so distinct that a "cat" in an image excites different neurons than a "cat" that I ask it about? Or is there overlap? Or we don't know :)
We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.
But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).
We only have very limited (but suggestive) evidence that the human brain has abstract "cat" neurons involved in various sensory and cognitive modes. Last time I paid attention, there was reasonably strong evidence that an image of a cat and reading the word cat used some of the same neurons. Beyond that it was pretty vague, there seemed to be evidence of a network that only weakly associates concepts from different modes, which isn't consistent with most people's subjective experience.
But then we have other evidence that what we think of as our subjective cognitive experience is at least partly a post-hoc illusion imposed for no apparent reason except that it creates an internal narrative consistency (which presumably has some utility, possibly in terms of having a mind others can form theories about more readily).