the machines arent optimized for it is why. the main driving factor is large unified memory which makes large(r) models possible, but there isnt the gpu horsepower to back it up. essentially, it fits the corner of the market that wants large models and is ok with running them slowly which doesnt sound like it would be a large market.
Macs have excellent generation speed, and the new Ultra will positively smash that at 1.2TB/sec of bandwidth. For example, that new 176B parameter Qwen model would generate tokens at ~200 tokens/sec.
Macs don’t have very good prefill, though. So it’s important to use a model serving stack that has excellent prompt caching and use a harness that won’t bust the cache.
I’m cross-shopping DGX Sparks and M5 Studios, and having a hard time deciding because they have exactly opposite characteristics for prefill and decode.
For decode, memory bandwidth is the main bottleneck, so these machines will likely perform well even without a ton of GPU horsepower. Not as well as Blackwell, but I expect they will be a reasonable choice in terms of price/performance if you want to run large models with a lot of context.
The main place they are a bit behind is in the number formats they support natively. Iirc M5 doesn't have native FP8 support, so you will take a speed penalty on quants where other architectures get better acceleration.
I really miss the prompting that A1111 had in concert with the grid. It was so easy to do a grids changing variables per axis. e.g. prompt on vertical and model on horizontal axis. is there a comfy node that does this? i've looked and havent found one.
> How do you decide what shared features a peacock and a palm tree have? At what scale should they be represented? How do you deal with invariance under affine transforms? Do you want to open the box of invariance under non-affine transforms?
The whole point is that the representation is learned. When you talk about various levels of abstraction, you're missing that all of these levels are representable with words and the relationships between them. That is verbatim what LLMs are optimized for. Interestingly, when you take an embedding, you do see that some transformations in embedding space actually hold which is quite interesting (e.g. tree + many ~ forest)
1) I am talking about representations beyond language models and language embeddings. If you take for example image, video, audio, 3D-spatial DICOM or combinations like VLMs. If you ask a language model to make an image of a Begonia ferox leaf without training it with images as well, it will not be able to represent this.
2) Language is already a higher-order lossy compressed abstraction made by humans to communicate fast and fill out the left out information with a learned prior. If you train a model on language only, it will not have the opportunity to have a non-compressed representation to make its own abstraction from.
3) If you are LLM-pilled and believe that we will be able to reach arbitrary levels of precise informational representation using language only, and that all abstractions that we may ever want can live on every single embedding layer in an LLM, your argument is fair.
1) If you ask a language model to make an image of a Begonia ferox leaf without training it with images as well
is this not true also of humans?
2) The idea that there is a non-compressed prior is already philosophical no? It seems to just be a rehash of whether an example of a "Chair" is a representation of a perfect Idea of a chair, or whether accidental things put together read as a chair. Its just dualism or not, Plato vs aristotle but for LLMs vs human learning.
If we do happen to just form an abstraction from less examples than an LLM, then its a question of efficiency in the learning algo and the capability to fill the gaps faster/better.
Obvs a world model and multi modal learning would close the gap between LLM latent space and what the human mind encodings might look like. But you can formulate a reality where humans do not have access to a non compressed representation (or even say that doesnt exist) and we are just more efficient at abstraction and recollection
3) There are multiple answers to this. One is that language can just be a prong in the learning modes available. Another is that language is many times redundant and includes supplemental information and nuances that despite being a higher order abstraction has enough additional information to reconstruct entire messages even after the lossy transmission.
Btw I am pretty against LLMs as path forward, but I have been interested in their ability to solve problems with Convergent solutions and their learning model as a lossy compression algorithm where instead of saving the actual data, you save a very good guesser of the next word. So if a text is part of its traning algo then you get almost a 1:1 (like a zip dictionary) but if its not, then he gives you something that plausibly could be in an imaginary Zip with that title.
This article reads like more of an ad than anything else.
> Environment variables only deliver values
Yes, the problem .env files try to solve is having "environment variables" be injectable from a file so that different applications can have different environment variables by default.
> A string is not a schema
Yes, input validation is an application concern. The application should know what these values represent / how to parse them and error if they're invalid.
I could go on, but the article is all about trying to use a hammer as a screwdriver and complaining that the hammer doesnt work.
> Yes, the problem .env files try to solve is having "environment variables" be injectable from a file so that different applications can have different environment variables by default.
This seems like a misunderstanding: The `.env` file should be `source`d into the current shell (environment). The application reads values using whatever mechanism it uses to read these values from the environment. Nothing should be “injected” into the application, i.e. the application should not read `.env` directly.
(Not sure if you were just being loose with your terminology, trying to clarify.)
Maybe. How does the application unify different environment variables? Those inherited by the shell, those read from .profile, those set in the process that start the application (be that ./run.sh, a nodejs script, systemd etc...).
Why would it want to? I'd say what you're asking for is actually a code smell. An app should not have a bazillion interlocking ways to be configured, unless there's a strong reason for this!
"AI is just a tool" is meant as a counterpoint to AI producing slop [1]. It isnt that there aren't ethical/societal implications with it as with everything we do in society.
[1] If you take the slop and manually fix/improve/verify it to production value it can be immensely valuable.
This is also how I have seen this particular thought terminating cliche used. The problem with the framing is that on the one side you have someone complaining about unrestrained slop and then this thought terminating cliche is offered. Why yes, it matters how you use it and the complaint is that users are not refining the output enough before presenting it to others.
The thought terminating issue with the phrase is that it isn't just a tool. Once you automate its use (automated PR reviews, ticket fixups, etc.), it becomes a process as well. You are almost certainly purchasing it as a service and not as a tool. It also contains elements of randomness that most tools do not. These quibbling points are what triggers the cognitive dissonance to produce the thought terminating effect. It is quibbling so you cannot really argue that point but the point is irrelevant; the second part of the cliche seems to be agreeing with the complaint about slop and yet the sentence is offered as some counterpoint. This termination prevents the actual conversation about how valuable its use is when the quality standard is not lowered.
This is a shallow analysis. Banter doesnt need to be about a transgression or offensive at all. E.g. if you were talking to Armand Duplantis, you could tease him about how easy he makes breaking the world record look: "Hey Armand, you havent broken the world record in this past year...arent you due?"
There is a difference between a service and a good.
It doesnt make sense to "own" a massage just the same way it doesnt make sense to "own" spectating a game in person. The video recording of people playing a sport is a good that you can own however. This is why an online/multiplayer game is harder to separate because it straddles the line of both a service and a good, but other cases are much more clear cut. (also, a quick google does reveal multipke open source diablo projects fyi)
I think you're confusing two different axes. There is a difference between the cache state and the context state.
Imagine a conversation with turns X, Y, and Z. When the LLM "reasons" about the next token A it does: P(A | X,Y,Z) and then P(B | X,Y,Z,A), etc. It will eventually produce a result P(D | X,Y,Z,A,B,C). Instead of continuing the context from X,Y,Z,A,B,C it continues it from X,Y,Z so you have P(N | X,Y,Z,D). This is what is meant by dropping the reasoning. This is done to save cache context for the session.
This is a different thing than preserving the K/V state of P(N | X,Y,Z,D).
No, I think the comment you're responding to is actually correct. Look at this quote from the Anthropic blog post again:
> The design should have been simple: if a session has been idle for more than an hour, we could reduce users’ cost of resuming that session by clearing old thinking sections. Since the request would be a cache miss anyway, we could prune unnecessary messages from the request to reduce the number of uncached tokens sent to the API. We’d then resume sending full reasoning history. To do this we used the clear_thinking_20251015 API header along with keep:1.
They clearly make the same distinction between the cache and the context. They're saying "we could reduce users’ cost of resuming that session by clearing old thinking sections". They intentionally created a behavior different between cached and uncached requests, specifically they clear thinking sections from the context for requests that miss the cache.
I hate these overly grand clickbaity statements. AI is a tool. You can use it well. You can use it poorly. "AI Slop" is the category of lazy AI tool usage. It is the same with AI code. Do you ask Claude to implement a feature and then not do a manual code review? If so, you're likely to get slop.
In this case use of an AI image, if people can clock it (which is super funny people are thinking "well, what if they can't tell!"), is the issue unto itself.
There is no using AI image generators _well_ if you care how people perceive you and your work.
> The research findings “could present a challenge to those who argue that the AI model does not store or reproduce any copyright works,” said Cerys Wyn Davies, an intellectual property partner at law firm Pinsent Masons.
The defense to training with copyright is that it is the same as how humans learn from copyrighted material. The storage or reproduction is a red herring. Humans can also reproduce copyrighted works from memory as well. Showing that machines can reproduce copyrighted material is no different than saying that a human can reproduce copyright material that the human learned from.
The defense to actually reproducing a work is that in order to do so, the user has to "break" the system. It is the same as how you can make legal software do illegal things (e.g. screen recorder to "steal" a movie)
None of this is to say that these defenses are correct/moral; but rather that this article doesnt add any additional input into whether it is or isnt.
> Humans can also reproduce copyrighted works from memory as well. Showing that machines can reproduce copyrighted material is no different than saying that a human can reproduce copyright material that the human learned from.
Ultimately this is a matter for the courts and the law, but I'd just like to point out that a human memorizing a work, reproducing it, and distributing it is just as much a copyright violation as doing a more mechanical form of reproduction.
There's a reason that fan fiction routinely falls afoul of copyright. There's quite a lot of case law in this area, and hand-waving "humans can do it too" doesn't really make for a strong argument. Humans get in trouble for it ALL THE TIME. The consequences can be fines, injuctions, or even criminal liability.
I'm not sure why you think AI gets off the hook here. Just because you like the outcome at the moment?
This isn't the defense you think it is. Performing a copyrighted work from memory - e.g. a piece of music, a poem, a story, etc - is still a copyright violation. There's no special protection for works that a human has memorized.
Humans are not judged on the basis of what they _can_ do.
Reasoning about how to constrain tools on the basis of what they _could_ do, if e.g. used outside their established guardrails, needs to be very nuanced.
Correct; the ability of a model to reproduce source material verbatim does not necessarily make the model's existence illegal. However, using a model to do just that might very well present a legal liability for the user. I would be interested to see the extent to which models can "recite from memory" source code, e.g., from the various MS code leaks. Put another way, if I'm using LLM code generation extensively, do I need to run a filter on its output to ensure that I don't "accidentally" copy large chunks of the Windows codebase?
>There's no special protection for works that a human has memorized.
Who's liable for the copyright infringement if you can coax it out of a system? If you can bypass paywalls by using google's cache feature (or since they got rid of it, but using carefully crafted queries to extract the entire text via snippets), is google on the hook or the person doing it?
Both. If I sell obviously pirated CDs on the street corner, it's not only illegal for me to copy them and sell them, it's also illegal for my customers to buy them.
Is it? There's plenty of people prosecuted for running illegal streaming sites and torrenting (which involves uploading), but I don't know of any efforts to crack down on non-distributors.
1. How does this interact with the ruling that both google books (ie. large scale scanning of books without author's consent) and google snippets (the same, but for websites) have been ruled legal by the courts?
2. Google might not be the most sympathetic defendant, but what about libraries? They offer books to be borrowed, and some offer photocopiers. If you put the two together, you get a copyright infringement operation, all enabled by the library. Should libraries be on the hook too?
For #2 yes...you would be engaging in copyright infringement. The library, being on the hook, would probably ask you to stop if they noticed you copying full books. If not the first time, certainly on the second
>If you can bypass paywalls by using google's cache feature
that is quite different. Google serves (used to serve) to its users whatever the website presents to its crawler, it does not try to avoid paywalls or interact with the website in any capacity other than requesting information
The whole “humans also do this” isn’t a winning defence here. Humans and copyright has long history and so much law that it is easy to get confused.
The default assumption here seems to be that the system needs to be broken. This is similar to the Google defence. If a user intent is to search for a cracked software what can poor Google do about it? The answer is to make it even more difficult.
This is a defence also used by torrent sites using magnet urls. “We don’t host files” is the default defence. But then if these sites get hit with DMCA they are required to remove the magnet url.
So the article shows what the lawyer is saying. Despite claims that it is difficult to search for full books, it really isn’t so. It is trivial. When it goes to court and it will, AI models will be required to make it even more difficult and allow for a DMCA like takedowns.
> Humans can also reproduce copyrighted works from memory as well
That's simply not true. No humans can memorize entire novels, as this research proved these models do. And definitely not all of these novels, and code bases, and who knows what else all at the same time.
>No humans can memorize entire novels, as this research proved these models do.
Humans can however, remember entire songs, and songs are definitely long enough to be considered copyright protected. There is still a difference in scale, but that's not really relevant when it comes to copyright law. You can't be like "well humans are committing copyright infringement but since it's limited to a few hundred words we'll give it a pass".
It's not that you can remember a song and therefore copyright infringement when you sing.
For 99.999% of people that are singing a song, it's not a replacement for the original in any way shape or form, hard stop. Let's not pretend it could even get anywhere close.
For the last 0.001%, we would call it a cover and typically the individually doing a cover takes some liberties of their own, still making it not a replacement in any way. Artists are typically cool with covers.
>For 99.999% of people that are singing a song, it's not a replacement for the original in any way shape or form, hard stop. Let's not pretend it could even get anywhere close.
You realize that lyrics are often written by someone other than the actual singer, and whoever wrote the lyrics is entitled to compensation too? The "amateur singing isn't a replacement for the studio album" excuse doesn't work in this context. Also courts have ruled that lyrics themselves are protected by copyright.
Clearly the team, if it is a team, that is entitled to the copyright is entitled to the copyright of the song, that's a silly statement to make. Copyright belongs to some entity, obviously.
You were specifically calling out individuals singing a song, not publishing lyrics online. These are not the same thing. Again your distribution/consumption model matters here.
On artists being "cool" with it - if the copyright holder doesn't pursue you then does it matter? The only valid argument I would see here is if the copyright holder doesn't know about the infringement and therefore cannot seek remedies, but we can fish for illegal scenarios all day if we would like: that's not useful though.
>Clearly the team, if it is a team, that is entitled to the copyright is entitled to the copyright of the song, that's a silly statement to make. Copyright belongs to some entity, obviously.
>You were specifically calling out individuals singing a song, not publishing lyrics online. These are not the same thing. Again your distribution/consumption model matters here.
I'm not sure why you're so confidently dismissive here. I wasn't trying to claim that nobody owned the lyrics. I brought that point up because even in the case of an amateur singing a song, even if you accept the "for 99.999% of people that are singing a song, it's not a replacement for the original in any way shape or form" excuse, you're still infringing on the copyright of the lyrics, because it's a derivative work. Moreover it's unclear whether that excuse even works. If you make a low cost version of star wars, copying the screenplay exactly, that still seems like copyright infringement, even if "it's not a replacement for the original in any way shape or form".
>On artists being "cool" with it - if the copyright holder doesn't pursue you then does it matter?
Virtually nobody got sued for torrenting with a VPN on. Does that mean it's fair to round that off as being legal, because "if the copyright holder doesn't pursue you then does it matter"?
> Moreover it's unclear whether that excuse even works. If you make a low cost version of star wars, copying the screenplay exactly, that still seems like copyright infringement, even if "it's not a replacement for the original in any way shape or form".
Are you being intentionally obtuse here? Intention matters here.
> Virtually nobody got sued for torrenting with a VPN on.
Let's not use obviously illegal actions which are done covertly to act as an example that is in any way similar to singing a song in the "open."
But the crime in the human instance is the reproduction, not the storage. So the crime in the AI circumstance would not be in the training, but in prompting the output.
And of course AIs are excellent at taking direction, so:
If I prompt it with "Harry Potter, but Voldemort wins: dark, and Hermione is a sex slave to Draco Malfoy" and get "Manacled," that's copyright infringement, and on me, not on the LLM/training.
If I prompt it with "Harry Potter, but Voldemort wins: dark, and Hermione is a sex slave to Draco Malfoy, and change enough to avoid infringing copyright," and get "Alchemised," then that should be fine. I doubt the legal world agrees with me though.
> But the crime in the human instance is the reproduction, not the storage. So the crime in the AI circumstance would not be in the training, but in prompting the output.
I wouldn't be so sure, at least under US law. 17 USC 101 defines a "copy" as:
[...] material objects, other than phonorecords, in which a work is fixed by any method now known or later developed, and from which the work can be perceived, reproduced, or otherwise communicated, either directly or with the aid of a machine or device.
If I memorize a work what ends up in my brain is not a copy according to that definition because with current technology there is no machine or device which can be used to perceive, reproduce, or otherwise communicate it. The work can only be perceived, reproduced, or otherwise communicated by using my brain which is not a machine or device.
No copy in my brain means that memorizing the work cannot infringe the copyright owner's exclusive right to reproduce the work in copies.
An LLM, unlike my brain, is a machine or device which can be used to perceive, reproduce, or otherwise communicate the work and so the work stored in the LLM is a copy.
Training an LLM then, unlike a brain memorizing a work, makes a copy and so would be covered by the copyright owner's exclusive right to make copies.
That's going to need to be justified, probably by arguing fair use.
I'd argue your brain is that "machine or device" -- the fact that the storage and the playback mechanism are one and the same is irrelevant. The fact that you have to be willing/induced to replay the content back just makes you a worse machine :-)
Interesting argument but not likely to go far. As far as I can tell US copyright law has never been taken to include brains as machines or devices.
This is actually relevant in some real cases, namely improvised works. Attempts to claim copyright on improvised works that were not recorded have generally failed. If brains counted as machines or devices than the work inside the performer's head would be a recording and the work would have copyright.
That is one of the reasons it is usually recommended that musicians should record their live performances. That gets them copyright on anything they improvise during the show. Also it gets them copyright on that particular performance of their music, which helps them go after anyone who makes an unauthorized recording of the show. (Copyright is only automatic upon recording when the recording is by or under the authority of the creator).
reply