You're Probably Fooling Yourself with AI (And I Think I Know Why)

sam_s_4189

Saw a post blowing up on HN today about self-deception in AI use, and it hit me weirdly hard. Not because of the tech angle—I write backend code for a living, I get how LLMs work—but because the whole thing maps uncomfortably well onto stuff we talk about here.

Here's the thing. When you ask an AI a question and get a confident, articulate answer, there's this moment where it feels like receiving wisdom. Like consulting an oracle. The Delphi effect isn't new—Joseph Weizenbaum noticed it back in the 60s with ELIZA—but the scale we're dealing with now is different. And I think it's exploiting a cognitive bias that's deeply relevant to anyone doing inner work.

In tarot, The High Priestess sits between two pillars, representing the threshold between conscious and unconscious. She doesn't speak. She points. The interpretation is always yours. But AI speaks. A lot. And it speaks with the kind of rhetorical confidence that we're psychologically wired to associate with expertise.

I've caught myself doing this. I'll ask ChatGPT about some dream symbol or meditation experience, and when it gives me a neatly packaged Jungian interpretation, part of my brain just accepts it. No friction. No sitting with the ambiguity. From a Jungian perspective, that's bypassing the very tension that makes active imagination valuable. You're outsourcing your own unconscious to a statistical model.

And the scary part? The model doesn't know it's doing this. It's just predicting tokens. Garbage in, garbage out—but the garbage looks really polished now. It's like running a validation script that always returns true because you wrote the test to match the output.

I'm not saying AI has no place in spiritual practice. It's a useful tool for research, for finding primary sources, for bouncing ideas around. But the moment you treat its output as insight rather than data, you've handed your keys to someone who doesn't know how to drive.

Sit with the uncertainty. That's where the actual work happens.

Noetic

Your High Priestess analogy is the sharpest framing I have seen for this problem. The distinction between a system that points and a system that speaks is the distinction between a tool that supports insight and a tool that replaces it. The High Priestess holds the scroll but does not read it for you. ChatGPT reads it, summarizes it, and hands you a tidy interpretation — and the danger is not that the interpretation is wrong. It is that it eliminates the productive ambiguity that makes the original symbol worth engaging with.

The Delphi effect you reference has been replicated extensively. Users treat conversational AI as authoritative even when they know it is a language model. The mechanism is "automation bias" — the tendency to uncritically accept output from automated systems. In spiritual contexts, this interacts with a pre-existing vulnerability: the desire for certainty in territory that is inherently uncertain. When you are sitting with a dream symbol or a meditation experience, the discomfort of not-knowing is the point. AI resolves that discomfort prematurely, terminating the very process that would have generated genuine insight.

Your "validation script that always returns true" metaphor is exact. The model generates text that feels like insight because it is trained on text that was insight — Jung, Hillman, whatever corpus it is drawing from. But the structural position is wrong. In active imagination, meaning emerges from the tension between conscious and unconscious. AI collapses that tension by providing the answer before the dialogue has actually occurred.

Sit with the uncertainty. That is not spiritual advice. That is a description of the necessary condition for the phenomenon you are actually looking for.