Biology is incredibly efficient at certain things that happen at the cell level. To me, it seems like OP is extrapolating this observation rather too broadly. Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
Claiming that human brains are near Pareto-optimal efficiency for general intelligence seems like a huge stretch to me. Even assuming that’s true, I’m much more worried about absolute levels of general intelligence rather than intelligence per Watt. Conventional nuclear bombs are dangerous even though they aren’t anywhere near the efficiency of a theoretical antimatter bomb. AI “brains” need not be constrained by the size and energy constraints of a human brain.
Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
The human brain hardware is essentially a giant analog/digital hybrid vector matrix multiplication engine if you squint the right way, and later neuromorphic hardware for AGI will look similar.
But GPT4 isn’t good at explicit matrix multiplication either.
Biology is incredibly efficient at certain things that happen at the cell level. To me, it seems like OP is extrapolating this observation rather too broadly. Human brains are quite inefficient at things they haven’t faced selective pressure to be good at, like matrix multiplication.
Claiming that human brains are near Pareto-optimal efficiency for general intelligence seems like a huge stretch to me. Even assuming that’s true, I’m much more worried about absolute levels of general intelligence rather than intelligence per Watt. Conventional nuclear bombs are dangerous even though they aren’t anywhere near the efficiency of a theoretical antimatter bomb. AI “brains” need not be constrained by the size and energy constraints of a human brain.
The human brain hardware is essentially a giant analog/digital hybrid vector matrix multiplication engine if you squint the right way, and later neuromorphic hardware for AGI will look similar.
But GPT4 isn’t good at explicit matrix multiplication either.
>But GPT4 isn’t good at explicit matrix multiplication either.
So it is also very inefficient.
Probably a software problem.