Besides how well something works, I am really curious if there’s any divergence that comes from different grammar in languages.
As in, the way languages are structured is different. Some are more precise, some are less, the information density per syllable is different, etc.
So besides just pure performance due to differences in training data, I’m curious if there’s some fundamental difference in the way LLMs interact with data in different languages even if end information is the same. Because even just in English, phrasing slightly different can yield different results.
Edit: would be interesting to see the “thinking” of the model done in different languages. Is identical problem thought about more or less the same, or does agent go on different train of thought depending on the language it is thinking in?
I am fairly convinced that there's a certain polyglot snowball effect: once the LLM is fluent in 20 languages, it can pick up on similarities in vocabulary, syntax etc. and learn the 21st language with much less effort (and training data). This might be difficult to actually study in an isolated way, but it's a real effect for humans and it makes sense the the pattern matchers that LLMs are would find these shortcuts.
Using similar words should land you in similar places in the latent space, even if they actual word or their order is slightly different. Where it gets interesting is how well English words map to their counterparts in other languages, and what practical differences it makes. From various studies, it seems that the gravitational pull of English language/culture training data is substantial, but an LLM can switch cultures and values when prompted in different languages.
Just saw your thinking edit! That's a great question and one I wanted to study in depth, but these days you don't really get access to the raw thinking data. It's usually summarized and you can't even be sure what language the model thought in unless you have access to the logits (so only viable for open-weights models).
Which also conveniently makes you spend more money on tokens.
With agile, at least no one was charging you for it. Like sure, there’s a cost to the process. But there wasn’t direct agile.com profiting from you.
Meanwhile agentic workflows every solution to the problem is giving more money to the ai companies.
Model is bad? Made more expensive model. Still bad? Here’s an infrastructure that reads huge text files again and again making you consume tokens. Still bad? Here’s a way to easily spin up multiple agents at once so you can delegate work. Still bad? Here’s a new service that will automatically review code. Still bad? Maybe a biggger more expensive model will help.
>> With agile, at least no one was charging you for it.
Depends. There are companies [1] making loads of money out of it. Charging for certification and imposing the idea that either you are certified, or you are going to fail. They are even eating the lunch of PMI, as PMI (PMBoK) is turning into an Agile manual.
Where I work is being expended literally millions per year in Agile.
> With agile, at least no one was charging you for it.
Charging people for Agile via his company ThoughtWorks (which sold for 785M) is how Neville Roy Singham made the money to fund far left groups in the US from his base in China.
As in, the way languages are structured is different. Some are more precise, some are less, the information density per syllable is different, etc.
So besides just pure performance due to differences in training data, I’m curious if there’s some fundamental difference in the way LLMs interact with data in different languages even if end information is the same. Because even just in English, phrasing slightly different can yield different results.
Edit: would be interesting to see the “thinking” of the model done in different languages. Is identical problem thought about more or less the same, or does agent go on different train of thought depending on the language it is thinking in?
reply