I suspect that the maximal utility of existence is somehow related to the highest level and diversity of insights similarly trained minds can produce (e.g. two copies of you would be interested in similar topics, and a substantial divergence would require a substantial divergence in memory describing the synapses or parameters). For example, the old GPT-2 had about 1.5B parameters and could only generate humanlike text, and the recently created GPT-5.4 (which, as far as I understand, activates ~400B parameters[1] and is a MoE) was reported to solve an open math problem. Similarly, I suspect that splitting a human brain, which has 100T synapses and is extremely undertrained, into ~100K bee brains having ~1B synapses each would likely prevent the bee brains from generating any insights more complex than those of GPT-2. However, I haven’t seen anyone attempt to make GPT-2-sized neuralese models solve complex problems.
- ^
To be precise, the authors of AI-2027 scenario predicted the appearance of GPT-5 using 2T params per token. The real models since GPT-5 were likely created by distillation of a model using 2T params/token and use 400B params/token.
If it wasn’t only Washingtonians who are to participate, then the number of recruits being two OOMs less than the 100K which the MIRI team demanded is less an evidence of the team’s incompetence (edit: or of political considerations that require the protest to be arranged in Washigton) and more of the position’s weakness. While the density of those who signed the pledge wasn’t disclosed, I suspect that the protesters would have to live close to the protest’s place. For example, I find it highly unlikely that, say, an army of people from states less eastern than Texas comes into Washington to protest and goes away after the protest is over.