How does anyone send these models that much context without it tripping over itself? I can't get anywhere near that much before it starts losing track of instruction.
I think it's key to not give it contradictory instructions, which is an easy mistake to make if you forget where you started.
As an example, I know of an instance where the LLM claimed it had tried a test on its laptop. This obviously isn't true so the user argued with it. But they'd originally told it that it was a Senior Software Engineer so playing that role, saying you tested locally is fine.
As soon as you start arguing with those minor points you break the context; now it's both a Software Engineer and an LLM. Of course you get confused responses if you do that.
The problem I often have is I may have instruction like-
General instruction:
- Do "ABC"
If condition == whatever:
- Do "XYZ" instead
I have a hard time making the AI obey the instances I wish to override my own instruction and without having full control of the input context, I can't just modify my 'General Instruction' on a case by case basis to simply avoid having to contradict myself.
I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts.
Right now I’m experimenting with using separate .plan files for tracking key instructions across domains like architecture and feature decisions.
> I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts.
This is the way. Not only have I had good luck with both a TASKS.md and TASKS-COMPLETE.md (for history), but I have an .llm/arch full of AI-assisted, for-LLM .md files (auth.md, data-access.md, etc.) that document architecture decisions made along the way. They're invaluable for effectively and efficiently crossing context chasms.
Yeah, this. Each project I work on has it's own markdown file named for the ticket or the project. Committed on the branch, and I have claude rewrite it with the "current understanding" periodically. After compacting, I have it re-read the MD file and we get started again. Quite nice.
