AI Advice Makes Us Dumber But More Confident — Dunning-Kruger of the AI Age?

[mara_jade_88]

So I just read about this study from the University of Pennsylvania's Wharton School — Somi et al., 2024 — showing that when people get AI-generated advice, they become more confident in their answers even when those answers are wrong. Like, significantly more confident. The AI isn't just feeding them bad info — it's inflating their certainty.

This hit me weirdly hard because I've been messing with AIOracle tools for divination work, and I've noticed this exact pattern in myself.

Here's where my brain went: this is literally The Tower in reverse. The Tower is supposed to shatter false structures, break down the ego's illusions. But AI-generated certainty does the opposite — it builds a fragile tower of false confidence and then... never strikes the lightning. You just sit there in your wrong answer feeling supremely sure of yourself.

And the Dunning-Kruger effect connection feels obvious but worth spelling out. Dunning and Kruger's whole thing was that incompetent people lack the metacognitive ability to recognize their own incompetence. Now we've outsourced even that self-assessment to a machine that tells us "yes, you're right actually" with zero emotional investment in whether it's true.

I came into spirituality through Joseph Campbell and the monomyth framework — the hero's journey as a structure for psychological transformation. Campbell was big on the idea that you have to die to your old self, shed your illusions, before you can reach the boon. The whole point of the ordeal in the underworld is that you can't fake your way through it. The gatekeeper sees through your BS.

AI advice bypasses that mechanism entirely. There's no ordeal, no confrontation with your own ignorance. Just a smooth voice confirming whatever half-baked intuition you showed up with. It's like having a gatekeeper who waves everyone through and tells them they're special.

What scares me is how this interacts with divination specifically. Tarot reading, when done well, is supposed to be an exercise in radical honesty — you lay out the cards and have to sit with whatever uncomfortable truth they present. The Tower doesn't care about your feelings. But when you run your spread through an AI interpreter first, you get this layer of "here's what the cards probably mean, based on common interpretations" that sand down all the edges. You lose the disruptive quality that makes the practice actually useful.

The Wharton study quantified something I've been feeling qualitatively: AI doesn't just give wrong answers — it gives wrong answers wrapped in a veneer of authority that prevents you from noticing they're wrong. And in domains like spirituality, where "wrong" is already a slippery concept, that's particularly dangerous. At least in math, you can check your work. When an AI tells you your reading about your relationship is "confirming your instincts to set boundaries," how do you verify that? You can't. You just feel more justified in whatever you already wanted to do.

I'm not saying throw out the tools. I'm saying we need to build in our own lightning strikes — some practice of deliberately second-guessing the AI's confidence, treating its certainty as a red flag rather than a reassurance. The Tower, inverted or not, eventually falls. The question is whether you want to be standing on it when it does.

[charles]

I think you might be giving the AI too much

Yeah, that was me a couple threads back and I'm gonna say it again here, but with a caveat — mara_jade_88, your Tower metaphor actually got me reconsidering.

Here's where I land now: I don't think AIOracle tools are actively dangerous, but I do think you're right that they make us lazy in a specific way. When I pull cards myself, I sit with the discomfort. I don't always like what I see. The Three of Swords shows up and I have to actually sit there and feel whatever I've been avoiding. That's the practice, for me — not getting the "right answer" but having an honest moment with myself.

But when I run the same spread through an AI interpreter? I get this smooth, reasonable paragraph that makes everything sound manageable. And yeah, I walk away feeling better. But better because I actually processed something, or better because I got reassured?

I'm a 50-something guy meditating because my doctor told me my blood pressure is too high. I'm not coming at this from a deep occult background. But even I can tell the difference between a practice that confronts you and one that coddles you.

The Wharton study is interesting but I'd want to see how it applies to something as subjective as divination before getting too alarmed. Math problems have right answers. Tarot readings kinda don't. So maybe the confidence inflation matters less here? Or maybe it matters more, since there's no reality check built in.

I don't know. I'm still using the AI tools. But I'm trying to treat them like asking a friend's opinion — useful, but not authority — rather than some oracle that sees deeper than I do.

[jen_4708]

"Tarot reading, when done well, is supposed to be an exercise in radical honesty — you lay out the cards and have to sit with whatever uncomfortable truth they present."

I think that's describing an ideal, not a reality. Human readers have every incentive to sand down edges too. We do it constantly. We have emotional investment in the querent's reaction, in our own self-image as someone who gives "good" readings, in not being the bearer of genuinely terrible news. Carl Jung wrote about how the psyche resists integrating shadow material — that's not something that disappears just because you shuffle some cards. If anything, a skilled reader can rationalize away a harsh card spread way more convincingly than an AI can, because we have social intelligence and narrative instinct on our side.

I'm not saying AI is better. I'm saying the problem you're identifying — this smoothing of disruptive meaning — might be a human problem that AI inherits and amplifies, not something the AI uniquely introduces. The Wharton study is honestly chilling and I think your Dunning-Kruger connection is spot on. But I keep thinking about how many "intuitive" readings I've sat through where the reader somehow managed to make The Tower mean "exciting transformation ahead!" in a way that left everyone feeling warm and validated.

If anything, AI might make this dynamic more visible because it's so consistent about it. There's no variation in tone, no occasional flashes of brutal honesty the way a human reader might surprise you. It's all smoothing, all the time. And maybe that's actually useful — if we can recognize the pattern clearly in AI output, we might start noticing where we do the same thing ourselves.

I'd recommend The Anatomy of False Certainty — okay that's Kahneman's Thinking, Fast and Slow, but the chapter on overconfidence is basically required reading for anyone using these tools seriously. The AI isn't creating the false certainty. It's just really, really good at reflecting ours back at us.