Four LLM Failure Modes and One User Failure Mode

Follow me on LinkedIn - AI, GA4, BigQuery

As I approach 1000 hours working with AI, I discovered 4 LLM failure modes and 1 user failure mode.

The five failure modes aren't edge cases or rare bugs. They're the structural reality of working with LLMs on any sustained technical task.

Anyone who doesn't understand them will lose work to them without knowing why.

Let's start with the most well-known LLM failure mode called ‘Hallucination’.

#1 LLM Failure Mode: Hallucination.

Hallucination is when the model produces content that isn't grounded in any source.

Common forms of hallucination:

Inventing citations and references.
Inventing facts ("the company was founded in 1987").
Inventing API behaviour ("this function takes an optional X parameter").
Inventing quotes from real people.
Inventing technical details that sound plausible.

For subject matter experts, hallucinations are the easiest to catch, as domain knowledge enables pattern recognition.

For non-experts, Hallucination is the most damaging because they can't catch it without domain knowledge.

#2 LLM Failure Mode: Overcorrection.

Overcorrection is when the model applies a valid principle beyond its scope of validity.

For example, you asked Claude to reduce a prompt's token size.

Claude did this by removing duplicate rules. The duplication looked like waste to Claude, but in the original prompt, it was intentional reliability redundancy (the same rule appeared twice because primacy plus recency catches the behaviour more reliably than either alone).

The principle Claude applied ("remove duplicate rules to reduce tokens") was correct. The application was wrong because Claude didn't check whether this specific duplication had properties that exempted it from the principle.

For subject matter experts, Overcorrection is often the hardest to spot because it produces plausible-looking output. To catch it, you must remember why specific rules or examples were added.

For non-experts, Overcorrection matters a lot less because they typically aren't asking the model to refine existing work.

#3 LLM Failure Mode: User-established context erosion.

User-established context erosion is when the model loses track of constraints, assumptions, decisions, or verified results that the user established earlier in the conversation.

Common forms of User-established context erosion:

Constraints the user set ("no em-dashes", "British English") slip back to defaults.
Assumptions established earlier get contradicted in later proposals.
Verified results from earlier testing get ignored.
Decisions reached through reasoning revert to the model's default position.
Corrections accepted in the moment return as the original wrong behaviour.

The model doesn't know its eroded context. You, as a user, must remember every prior constraint, assumption, verified results, decisions reached and corrections applied.

For subject matter experts, Context erosion is the second hardest to catch (after overcorrection) because you must remember at all times what you and your AI Assistant have established so far.

You can't rely on LLM for this purpose.

#4 LLM Failure Mode: Model-default context erosion.

Model-default context erosion is when the model loses track of its own system-level context (its style preferences, its default behaviours, its self-knowledge about how it should operate).

When the model's default context erodes, the model reverts to its training data.

For example, your system prompt says "use markdown formatting." Early in the conversation, the model follows this. By turn 40, the model is using headers and bullets everywhere. The system instruction is still there. The model stopped weighting it.

This is the same mechanism as user-established context erosion, but it affects the model's defaults rather than user-established context. Both forms erode because attention to early context degrades as the conversation grows.

Even for subject matter experts, it is often not easy to spot when the LLM starts violating its system prompt.

#5 User mental fatigue.

It's the cumulative cognitive cost of reading the model's output, evaluating it for correctness, comparing it against earlier context, and verifying the changes the model made and that too over and over again until your task/project is completed.

You ask the model to clean up a prompt. The model produces a cleaned version, a change summary, and a flag list. You read all three carefully, catch several overcorrections, and push back.

The model produces a revised version with new explanations and new flags. You read everything again. Each round adds reading load.

By the 10th round, you are skimming rather than verifying. And just one, you get lazy, the model's next overcorrection slips through.

User fatigue scales with work length. The longer the session, the more it shapes which other failures land.

The dominant failure mode also depends on what you are currently doing:

One-shot factual questions → hallucination dominates.
Cleanup and refinement of existing work → overcorrection dominates.
Long iterative conversations → context erosion and user fatigue dominate.
Building production systems with LLMs → overcorrection, context erosion, and user fatigue all dominate.

So, what can you do?

I don’t know about others. But I maintain extensive documentation/versions of the important iterations. So I can easily revert back to the last known good configuration, as they say.

If I find myself skimming rather than reading the LLM outputs, I take a break or do something else that doesn’t require that level of cognitive load.

Version control seems to handle the consequences of overcorrection and context erosion.

If the model produces a cleaned version that turns out to have removed something load-bearing, you revert to the last known good version.

You don't have to be perfect at catching overcorrection in real time, because the wrong changes don't compound, each one is reversible.

The same applies to context erosion. If the model proposes work that contradicts something established earlier, you don't lose anything by accepting the proposal and then reverting if it breaks. The version history is the safety net.

However, in my opinion, the strongest safety net is awareness of these 5 failure modes.