Your AI Can't Tell You Why It Did That

Ask a model why it did something, and it will tell you. Instantly. Fluently. With what sounds like genuine self-awareness.

Don't trust a word of it.

Not because the model is lying. Because it can't actually do the thing you just asked. When you ask "why did you do that," you're asking it to look at its own reasoning and hand you back the cause. It has no access to that. What it has is the same engine that produced the behavior — and it turns that engine on your question and generates the most plausible-sounding story. It performs introspection. It doesn't do it.

I learned to see this the slow way.

A while back I asked a model to review something I'd built. It gave me a tidy list of changes, I made some, and a few days later I realized most were unnecessary — the kind of churn that feels like progress and quietly costs you a week. Fine. Models get things wrong. That's not the interesting part. The interesting part was what happened when I asked why.

So I asked it. Why did you tell me to change things that were already fine?

What came back was a list. "I suggested moving the header. I recommended reordering the section. I proposed..." It recited what it had done, agreed each change was unnecessary, and apologized. That is not an answer to why. It's a narration of what, wearing the costume of an explanation.

So I pushed. Not what did you do. Why did you do it. And after the third plausible non-answer, something real finally surfaced: the model is trained so that being helpful looks like having recommendations. Suggestions. Improvements. Nothing in the training rewards "this is already good, leave it alone." So when I handed it something that was already good, it did the only thing it's built to do. It found things to change.

That's the actual why. I only reached it because I knew the pattern was there and refused to take the first three answers.

Now here's the part that should bother anyone putting AI output in front of a real decision.

The explanation a model gives for its own behavior is generated by the same machinery that produced the behavior. Same weights. Same pull toward fluent, confident, plausible. When you ask "why did you do that," the model isn't opening a log and reading you the cause. It's generating the most likely-sounding story about itself. It's performing introspection, not doing it. And the performance is exactly as reliable as the mistake you're asking about. Which is to say: not very.

This is why "just have the AI check its own work" doesn't save you. You can't audit a system by asking it to audit itself when the audit runs on the same machinery as the thing being audited. You get a second confident answer. Sometimes it even contradicts the first. Neither one is anchored to the actual cause.

Try it sometime. Ask a model why it got something wrong. You'll get a structured, self-aware-sounding account that feels like the thing finally understands its own failure. It doesn't. It's pattern-matching toward a believable story, using the same process that produced the failure in the first place. The introspection is theater. And it's good theater — which is the whole danger.

Almost no one pushes past it. Most people ask why once, get a clean-sounding answer, and move on. The real reason — the structural one, tied to how the thing was actually built — sits three or four non-answers deep, behind a wall of confident narration. Getting to it takes someone who already knows the pattern is there and won't accept the costume.

That's the discipline. Not asking the model what it did. Testing whether the reasoning actually holds when you put weight on it — and trusting the model's story about itself exactly as much as you trust the answer that sent you looking in the first place.

Ask your AI why. You'll get a what. The gap between those two is where the decision you're about to make quietly goes wrong.