Rafael Rayes

Obviously, It Is Not Conscious

If an extremely intelligent alien opened up and studied a human brain, it would say there is nothing mystical in there, just xulerkzion patterns.

A linear regression is, in principle, not very different from a complex neural network. For the purposes of this discussion, they are the same. Train a regression to predict the price of a house on the California housing dataset, and explaining the model is extremely simple. By looking at each of the parameters we can easily see that the model gives more importance to the size of a house than to its street number. The reasons why the algorithm makes any prediction are perfectly clear to us: just look at the weight assigned to each variable.

Take a slightly more complex neural network, a digit classifier for example. The task of explaining the reason for each decision is harder, but still reasonable. Observing the activation of each neuron, mapping areas of the input to results in the output, all of that is possible.

An animal brain works analogously to this neural network. The simpler the brain, the easier it is to understand causes and consequences, to explain how the brain works and how the animal behaves. Take the brain of a fly. A lot of technology is needed, but it is still possible; the question is feasibility. It is easy to understand: the wind hits the wing, activating nerves xyz, the electrical signal propagates to area qzn of the brain, and the way those neurons are connected generates an output that goes back to the nerve. Dendrite! Myelin sheath, acetylcholine, sarcoplasmic reticulum, calcium pump! Boom… the fly’s legs contract, similar processes keep happening, and the fly flies. Obviously, it is not conscious!

As the circuits expand and become more complex, the behaviors diversify. Explanations get harder and harder to find. Even so, it is possible to identify causes and consequences: the dog sits, gets a biscuit, releases dopamine, the neural pathways are consolidated. It is not conscious! At each step, our explanations are progressively obscured behind one more layer of abstraction.

But here is the question: do you need to understand the mechanisms underneath in order to get a result? At the moment AlexNet shocked the world, little was known about the mechanisms behind its seemingly miraculous classifications. The work of explaining AlexNet came afterwards.

The human brain. Perhaps the most mysterious object of study ever contemplated by man, a system that, singularly, investigates itself. The mechanisms responsible for human behavior can be explained by the same principles we saw in the fly. But just as the simple linear regression is equivalent to the fly, and AlexNet is equivalent to the dog, the circuits that govern a human being are exponentially more complex. Any explanation becomes a task for gods.

Modern LLMs are, to some extent, remotely comparable to human beings in terms of explainability. But unlike our brains, one of these neural networks is something we are perfectly able to examine, dissect and watch in action. And still, not even the creators of the model knew for sure what was happening that made it work:

A limitation, or at least uncertainty, associated with few-shot learning in GPT-3 is ambiguity about whether few-shot learning actually learns new tasks “from scratch” at inference time, or if it simply recognizes and identifies tasks that it has learned during training. […] Ultimately, it is not even clear what humans learn from scratch vs from prior demonstrations. […] understanding precisely how few-shot learning works is an important unexplored direction for future research.

Brown et al., “Language Models are Few-Shot Learners”

More recent studies of the mechanisms behind the emergent capabilities of these models have been shedding light on the subject. It is nothing more than geometric patterns. They are not conscious!

Translated from a note I wrote in Portuguese in March 2026.