Ehh... can't really hit "chatbot" limits on the $20 plan. Pretty sure the limits are not token based for that in the first place, and if it spews out a ton of stuff, it takes me longer to go through it and I end up asking it follow-up questions in a way where it replies... _relatively_ concisely. Still, gimme robot back. On a good note, it almost managed to call me stupid.
Codex has also been fine, but I'm guessing they know better than to tweak it like that, given their target users.
I have hit chatbot limits with the $20 a month plan. During the day I use it with Codex and I night I use it to study Spanish. I don’t know if the two are correlated.
But then I just switch to another OpenAi and strangely enough, chat forces me into “thinking mode” when that happens and won’t let me do instant
Re: $1k/day on tokens - you can also build a local rig, nothing "fancy". There was a recent thread here re: the utility of local models, even on not-so-fancy hardware. Agents were a big part of it - you just set a task and it's done at some point, while you sleep or you're off to somewhere or working on something else entirely or reading a book or whatever. Turn off notifications to avoid context switches.
On that note, I could also comfortably fit a couple of chat windows (skype) on a 17'' CRT (1024x768) back in those days. It's not just the "browser-based resource hog" bit that sucks - non-touch UIs have generally become way less space-efficient.
FoundationDB's approach - look up their testing framework.
I've worked in a company that, for all intents and purposes, had the same thing - single thread & multi process everything (i.e. process per core), asserts in prod (like why tf would you not), absurdly detailed in-memory ring buffer binary logs & good tooling to access them plus normal logs (journalctl), telemetry, graphing, etc.
So basically - it's about making your software debuggable and resilient in the first place. These two kind of go hand-in-hand, and absolutely don't have to cost you performance. They might even add performance, actually :P
Re: "yes men" - critical thinking always helps. I kind of treat their responses like a random written down shower thought - malicious without scrutiny. Same with anything that you haven't gone over properly, really.
The advantages that you listed make them worth it.
The output of the prompts always needs peer review, scrutiny. The longer is the context, the longer it will deviate, like if a magnet were put nearer and nearer to a navigation compass.
This is not new, as LLMs root are statistics, data compression with losses, It is statistically indexed data with text interface.
The problem is someones are selling to people this as the artificial intelligence they watched at movies, and they are doing it deliberately, calling hallucinations to errors, calling thinking to keywords, and so on.
There is a price to pay by the society for those fast queries when people do not verify such outputs/responses, and, unfortunately, people is not doing it.
I mean, it is difficult to say. When I hear some governments are thinking in to use LLMs within the administrations I get really concerned, as I know those outputs/responses/actions will nor be revised nor questioned.
Characteristics - Defaults (they must've appeared recently, haven't played with them)
Custom instructions:
"Be as brief and direct as possible. No warmth, no conversational tone. Use the least amount of words, don't explain unless asked.'
I basically tried to emulate the... old... "robot" tone, this works almost too well sometimes.
Same with doing things in RAM as well. Sequential writes and cache-friendly reads, which b-trees tend to achieve for any definition of cache. Some compaction/GC/whatever step at some point. Nothing's fundamentally changed, right?
They recently made "efficient" even more verbose, my custom instructions can't suppress it properly anymore.
These "little" changes are incredibly annoying.
reply