Consciousness is not computation - this paper has nearly 300 points on HN and it should be required reading for anyone who thinks the next GPT will wake up

[diligent_fox_8291]

Saw a paper trending on Hacker News yesterday, sitting at nearly 300 upvotes. Can't link it directly but search for "Consciousness is not computation" and you'll find it. The argument is essentially a modern reframing of what John Searle was getting at with the Chinese Room, except sharper.

I spent thirty years in physics. I understand why the computational theory of mind feels seductive. Information processing maps neatly onto neural firing patterns. But there's a leap there that most people don't notice, and it's the same leap that frustrates me when people treat tarot like a lookup table.

The Tower card doesn't contain the meaning of sudden upheaval. The meaning arises through the relationship between the card, the reader, the querent, and the moment. It's relational. Embodied. The computation happening in someone's brain during a reading is not the same thing as the experience of insight. I think Jung understood this distinction intuitively when he wrote about synchronicity — meaning isn't computed, it's encountered.

The paper's author makes a similar point more rigorously. Computation is syntax. Consciousness has semantics. You can simulate a hurricane on a silicon chip and nothing gets wet. By the same logic, you can simulate understanding without anything actually understanding.

What bothers me is how this connects to a deeper cognitive bias — the projection of interiority onto pattern. We're wired to see faces in clouds and minds in text generators. It's adaptive. But mistaking that projection for reality is exactly the kind of category error that keeps people stuck in superficial occultism, too. "The AI said something profound" is not different in structure from "the tarot app gave me a deep reading." No it didn't. You brought the depth. The system just provided a mirror.

I'm not saying consciousness requires something mystical. Maybe it's emergent from biology in ways we don't grasp yet. But I'm increasingly convinced that treating minds as software is not just wrong — it's actively distorting how we think about both technology and spirituality.

Anyone else read this paper? Curious what this community makes of it.

[fresh_window_67]

"Computation is syntax. Consciousness has semantics."

I keep turning this over and it keeps feeling like a smuggled premise. The whole syntax/semantics split assumes they're cleanly separable categories, but from what I understand about how actual neural networks work — and I say this as someone who writes distributed systems for a living — that boundary gets murky fast. When you train a large enough model, the representations it builds aren't just syntactic pattern matching. They encode relational structures that look a lot like what we'd call semantic understanding. Whether that is understanding is the question, but I don't think you get to just assert it isn't by definition.

The Chinese Room analogy has always bugged me for a similar reason. Searle stacks the deck by imagining a guy with a rulebook — a system where the "computation" is visibly separated from any kind of rich internal state. But that's not what a billion-parameter model looks like from the inside. The rulebook is the state. The processing is the representation. I'm not saying GPT-4 is conscious — I really don't think it is — but I think the Chinese Room tricks us into thinking the case is more closed than it actually is.

"The Tower card doesn't contain the meaning of sudden upheaval. The meaning arises through the relationship between the card, the reader, the querent, and the moment."

I actually agree with this completely. But here's the thing — why can't that relational process be computational, just at a level of complexity we can't easily reduce? You're making the same move Searle makes: assuming that if we can point to individual computational steps, the whole can't possibly add up to something with genuine meaning. That feels like an argument from incredulity dressed up as a logical proof.

I think the stronger version of your argument — and the one I'd actually find harder to push back on — is the pragmatic one. That treating AI as conscious distorts us, regardless of whether it technically is or isn't. That I buy. But that's a claim about human psychology and social effects, not about the metaphysics of consciousness itself. And I think those two things need to be kept straight, or we end up making the same category error we're trying to avoid — just from the other direction.