I was using KeepChatGPT[1] for a while back in 2023-2024, pre-Gemini-in-Google era, and I was fascinated as to how it was able to mask being a user without needing any API or help from the end user. I stopped using it after 2024 because 1) Gemini and 2) It breaks quite a lot. I did however, like how you had an option to push the AI panel to the right, if only Google even considers doing so.

[1]: https://github.com/xcanwin/keepchatgpt

I have a little helper app I run sometimes that I have a button to push a query into ChatGPT and get a json response. You wouldn't even know OpenAI had any anti-bot tools because it doesn't get flagged at all. It just uses a webview inside WinForms.

I also genuinely hate using jq. It is one of the only things that I rely heavily on AI.

You should try nushell or PowerShell which have built ins to convert json to objects. It makes it so easy.

Second this. Working with nushell is a joy.

I use the llm-jq plugin for Simon Willison's `llm` command line frontend for this: https://github.com/simonw/llm-jq

At that point why don't we ask the AI directly to filter through our data? The AI query language is much more powerful.

Because the output you get can have hallucinations, which don’t happen with a deterministic tool. Furthermore, by getting the `jq` command you get something which is reusable, fast, offline, local, doesn’t send your data to a third-party, doesn’t waste a bunch of tokens, … Using an LLM to filter the data is worse in every metric.

Just because you haven't seen it hallucinate on these tasks doesn't mean it can't.

When I'm deciding what tool to use, my question is "does this need AI?", not "could AI solve this?" There's plenty of cases where its hard to write a deterministic script to do something, but if there is a deterministic option, why would you choose something that might give you the wrong answer? It's also more expensive.

The jq script or other script that an LLM generates is way easier to spot check than the output if you ask it to transform the data directly, and you can reuse it.

> but if there is a deterministic option, why would you choose something that might give you the wrong answer?

Claude Code can use jq if it's installed on your system. Also, the data transformation is usually part of a larger workflow where an LLM is being used. And honestly, Claude is going to know jq better than 95% of developers who use it. jq can do a lot of things but it’s not the most intuitive tool to learn.

An obvious best practice is to have the LLM use existing tools to confirm the correctness of its output.

LLMs will often helpfully predict made up tokens for the content of the data fields.

For 100% of jq use cases I have the data wouldn’t fit into context. But even for the smaller things, I have never, not even once, had an LLM not mangle data that is fed into it.

Take a feed of blog posts (and select the first 50 or so just to give the model a fighting chance). I’ll give you 80% likelihood of the output being invalid JSON. And if you manage to get valid JSON out of it, the actual dates, times and text content will have changed.

"so it is faster" than what? A cloud hosted LLM? That's a pretty low bar. It's certainly not faster than jq.

There is hardware that is able to run jq but no a local AI model that's powerful enough to make the filtering reliable. Ex a raspberry pi

Because the input might be sensitive.

Because the input might be huge.

Because there is a risk of getting hallucinations in the output.

Isn't this obvious?

...and because it's going to burn a million times the energy of what jq would require.

You need to learn to adapt to the real world where most things are not deterministic. Go and learn.

I already know that. That's why we have deterministic algorithms, to simplify that complexity. You have much to learn, witty answers mean nothing here, particularly empty witty answers, which are no better than jokes. Maybe stand-up comedy is your call in life.

That may be true, but do you not want determinism where possible, especially within this context, i.e. filtering data?

Is your argument that the world isn't deterministic and so we should also apply nondeterminism to filtering json data?

> VERY verbose in the plan output

Is that an issue? GitHub charges per-request, not per-token, so a verbose output and short output will be the same cost

What model are you using?

The problem might be that our brains charge per token, which makes reviewing hard. :)

The post you linked to was made after this post.

my bad, I just saw 2 hours ago on this one - either way they could be merged

I think my submission about this post was selected to "second-chance" pool by HN Moderators. Hence it's being shown again. Thanks for the heads up!

Nope, check out something like wiretext, look at this example I put together very quickly

https://wiretext.app/w/WUtjS1bk

That literally is the definition of JIT, it does a quick parse, compiles hot parts and improves it later on

That is the issue. It's why Xcode development is really bad with AI models[0] -- because there are barely any text-based tutorials for it, so the models have to make a lot of assumptions and whatnot. Hence, they are really good at Python, JavaScript, and increasingly, Rust.

[0]: https://www.youtube.com/watch?v=J8-CdK4215Y

This is very very concerning, and I really hope he doesn't fall depper into this.

I've been having issues logging into my HN account on Harmonic for quite a while, now this, the API is down :/

It's just their tech debt. They chose to use react.js in the terminal via ink, and essentially footgun their way through each update.