No, it's not a linear regression, but it is at its heart optimization.
And yes, that is indeed like saying a computer is just a bunch of zeroes and ones and logic gates. It's true and beautiful and profound but nearly useless from a practical perspective. And the wonder, like computers when you scale the numbers of transistors to the billions, is that when you scale the number of parameters to the billions, you end up with something amazing.
Yeah but the people making these comments aren’t trying to point to the wonder of how simple mathematics can underpin large complex systems, they’re doing the opposite: Trying to trivialize the system and its immense potential for good and bad by pointing out that it uses simple mathematics under the hood (and thus can’t be that amazing).
No, they're not trying to deny the system is amazing. They're saying that it's not magic, and that we will soon be able to create comparably amazing systems ourselves.
And yes, that is indeed like saying a computer is just a bunch of zeroes and ones and logic gates. It's true and beautiful and profound but nearly useless from a practical perspective. And the wonder, like computers when you scale the numbers of transistors to the billions, is that when you scale the number of parameters to the billions, you end up with something amazing.