i had the feeling that Karpathy is not a software developer at all, reading his tweets is like he is discovering half a century ago ideas. like specification...
He's not, he was an AI researcher and then led AI research teams.
He's doing "engineering" now since he can use LLMs, but he never really spent years doing what we would call software engineering. I.e. building distributed systems, writing terraform/ansible, maintaining old databases or optimizing MySQL indexes, debugging Kafka or AzureMQ, etc.
Don't get me wrong, he's knowledgeable about LLMs specifically, (although his knowledge is rapidly becoming out of date) but he's not a software engineer, which is why his ideas around engineering often seem totally deluded.
what is sad, is that you an take the article, go to any LLM, ask to read and build an system prompt, from the images + text, and then you can replicate his work
I think what's more sad is someone going out of their way to NOT support the artist.
Art is, among other things, a conversation starter. If someone sees an LLM generated thing hanging in the home or office and strikes up a conversation, it goes something like: "Yeah, i saw it online and I ended up ripping it off"...or the person lies to save face (also sad).
Or, the conversation could be "Yeah, these were purchased directly from the artist...I bought them because....". (a much more interesting conversation)
Both of these evoke an emotional reaction with an interlocutor.
You got your reply already. To add: YJIT is the one that does "basic block versioning" (Which was Maxime's thesis) while ZJIT is a more traditional design.
I am confident in that description but don't actually know what it means in practice (yes I've seen papers and talks, but I kinda need not-compiler-engineer to explain it to me.)
As I understand it BBV still holds promise, but the sheer volume of knowledge of more traditional methods might mean it gets better outcomes (also IIRC ZJIT is still lagging YJIT).
reply