Follow me on LinkedIn - AI, GA4, BigQuery

So I want to tell you about something weird that happened while I was building a voice agent this week, because I think it's a bigger problem than people realise.

Everyone's worried about AI hallucinating facts. Making up a stat, inventing a quote, whatever. Fair enough.

But there's a different kind of hallucination that nobody talks about, and it's way more dangerous if you're actually shipping something: an LLM hallucinating edits.

If you're using Claude (or any LLM) to maintain your agent prompts, don't ask for the "full updated prompt" every time.

Because if you do, your LLM may fix one thing and silently break something else while confidently telling you that its edits did not result in any loss of behaviour. Nothing behavioural changed: same rules, same scripts and same flow.


You get the entire prompt back, rewritten with a list of all edits made by the LLM. You believe those edits. They looked fine on a skim, but the agent still failed in production.

Now you don't know what your LLM accidentally changed without notifying you. You tried to fix one thing and ended up breaking something else.


→ Ask for the smallest possible change — one line, one clause, exactly where it goes.


→ Paste that into your own master copy yourself.

→ Never let the model regenerate the whole file as "the update."


Full-file regenerations feel thorough. What they actually do is often bury every accidental edit inside a wall of text that looks unchanged at a glance. You lose your diff. You lose your ability to review. You lose your ability to revert.

Your prompt file is the source of truth. Claude should be handing you patches, not rewrites.

Small edits. Manual merge. Every time.