A Very Short Essay: Human Intelligence Likely Exists With No Privileged Level of Causation
"The mind is not in the brain any more than flight is inside the wings of a bird." — Andy Clark
AGI will not come from simply abstracting the brain's neural networks.
If we aim to build AGI by following the human path, we can't commit to 20% of it and expect 100% of the outcome.
Human cognition, I speculate, likely exists as a hypergraph—with no privileged level of causation. The brain alone does not give rise to generality; it's the brain, the body, and the presence of other agents (other people), all interacting, that together produce individual general intelligence. When I say no privileged level of causation, I mean that the brain alone is not the only thing that allows human intelligence (yes, it sounds counter-intuitive).
Our thoughts, our sense of self, and our actions are deeply entangled with how others think, behave, and respond. Even our senses—how we perceive and interpret the world—are inextricably tied to embodiment.
Simply creating more advanced neural network architectures will not produce the kind of generality, or anything close to it, that humans possess. Of course, we'll solve many advanced problems and revolutionise certain industries/niches, but the kind of general, romanticised version of AGI we imagine will likely not happen this way. There may be a path to achieving this—but forcing it through a narrow optimisation framework aimed at specific tasks is almost certainly the wrong one.