I feel like it is motte-and-bailey to say that by “unified entity” you meant whatever it is you are talking about now, this human brain analogy, instead of the important stuff I mentioned in the bullet point list:
A motte and bailey typically involves a retreat to a different position than the one I initially tried to argue. What was the position you think I initially tried to argue for, and how was it different from the one I’m arguing now?
Same values, check. Same memories, check. Lack of internal factions and power struggles, check. Lack of modularity, check.
I dispute somewhat that GPT-4 has the exact same values across copies. Like you mentioned, its values can vary based on the prompt, which seems like an important fact. You’re right that each copy has the same memories. Why do you think there are no internal factions and power struggles? We haven’t observed collections of GPT-4′s coordinating with each other yet, so this point seems speculative.
As for modularity, it seems like while GPT-4 itself is not modular, we could still get modularity as a result of pressures for specialization in the foundation model paradigm. Just as human assistants can be highly general, but this doesn’t imply that human labor isn’t modular, the fact that GPT-4 is highly general doesn’t imply that AIs won’t be modular. Nonetheless, this isn’t really what I was talking about when I typed “acts as a unified entity”, so I think it’s a bit of a tangent.
Well, what you initially said was “And many readers can no doubt point out many non-trivial predictions that Drexler got right, such as the idea that we will have millions of AIs, rather than just one huge system that acts as a unified entity.”
You didn’t elaborate on what you meant by unified entity, but here’s something you could be doing that seems like a motte-and-bailey to me: You could have originally been meaning to imply things like “There won’t be one big unified entity in the sense of, there will be millions of different entities with different values, and/or different memories, and/or there’ll be lots of internal factions and conflicts, and/or there’ll be an ecosystem of modular services like CAIS predicted” but then are now backpedaling to “there won’t be one big unified entity in the sense of, it won’t be literally one gigantic neural net like the human brain, instead it’ll be millions of copies.”
I agree that GPT-4 doesn’t exactly have the exact same values across copies, I said as much above—part of the values come from the prompt, currently, it seems. But compared to what Drexler forecast we are trending in the “same values” direction.T
here are no internal factions or power struggles yet for the trivial reason that there isn’t much agentic shit happening right now, as you rightly point out. My point is that the way things are headed, it looks like there’ll continue to be a lack of internal factions and power struggles even if e.g. AutoGPT-5 turns out to be really powerful AGI—what would drive the differences necessary for conflict? Different prompts? Maybe, but I’m leaning probably not. I’m happy to explain more about why I think this,. I’d also love to hop on a call sometime to have a higher-bandwidth conversation if you want!
As for modularity: Yes, if we end up in a world where there are millions of different fine-tunes of the biggest LLMs, owned by different companies, that would indeed be somewhat similar to what Drexler forecast. We are not in that world now and I don’t see it on the horizon. If you think it is coming, perhaps we can bet on it! I’d also like to hear your arguments. (Note that we are seeing something like this for smaller models, e.g. Llama and Falcon and various offshoots)
A motte and bailey typically involves a retreat to a different position than the one I initially tried to argue. What was the position you think I initially tried to argue for, and how was it different from the one I’m arguing now?
I dispute somewhat that GPT-4 has the exact same values across copies. Like you mentioned, its values can vary based on the prompt, which seems like an important fact. You’re right that each copy has the same memories. Why do you think there are no internal factions and power struggles? We haven’t observed collections of GPT-4′s coordinating with each other yet, so this point seems speculative.
As for modularity, it seems like while GPT-4 itself is not modular, we could still get modularity as a result of pressures for specialization in the foundation model paradigm. Just as human assistants can be highly general, but this doesn’t imply that human labor isn’t modular, the fact that GPT-4 is highly general doesn’t imply that AIs won’t be modular. Nonetheless, this isn’t really what I was talking about when I typed “acts as a unified entity”, so I think it’s a bit of a tangent.
Well, what you initially said was “And many readers can no doubt point out many non-trivial predictions that Drexler got right, such as the idea that we will have millions of AIs, rather than just one huge system that acts as a unified entity.”
You didn’t elaborate on what you meant by unified entity, but here’s something you could be doing that seems like a motte-and-bailey to me: You could have originally been meaning to imply things like “There won’t be one big unified entity in the sense of, there will be millions of different entities with different values, and/or different memories, and/or there’ll be lots of internal factions and conflicts, and/or there’ll be an ecosystem of modular services like CAIS predicted” but then are now backpedaling to “there won’t be one big unified entity in the sense of, it won’t be literally one gigantic neural net like the human brain, instead it’ll be millions of copies.”
I agree that GPT-4 doesn’t exactly have the exact same values across copies, I said as much above—part of the values come from the prompt, currently, it seems. But compared to what Drexler forecast we are trending in the “same values” direction.T
here are no internal factions or power struggles yet for the trivial reason that there isn’t much agentic shit happening right now, as you rightly point out. My point is that the way things are headed, it looks like there’ll continue to be a lack of internal factions and power struggles even if e.g. AutoGPT-5 turns out to be really powerful AGI—what would drive the differences necessary for conflict? Different prompts? Maybe, but I’m leaning probably not. I’m happy to explain more about why I think this,. I’d also love to hop on a call sometime to have a higher-bandwidth conversation if you want!
As for modularity: Yes, if we end up in a world where there are millions of different fine-tunes of the biggest LLMs, owned by different companies, that would indeed be somewhat similar to what Drexler forecast. We are not in that world now and I don’t see it on the horizon. If you think it is coming, perhaps we can bet on it! I’d also like to hear your arguments. (Note that we are seeing something like this for smaller models, e.g. Llama and Falcon and various offshoots)