I like your distinction between narrow and broad IC dynamics. I was basically thinking just about narrow, but now agree that there is also potentially some broader thing.
How likely do you think it is that helper-nanobots outcompete auto-nanobots? Two possible things that could be going on are:
I’m unhelpfully abstracting away what kind of AI systems we end up with, but actually this significantly impacts how likely power concentration is and I should think more about it
Theoretically the distinction between helper and auto-nanobots is significant, but in practice it’s very unlikely that helper-nanobots will be competitive, so it’s fine to ignore the possibility and treat auto-nanobots as ‘AI’
Yup sorry, the Tom above is actually Rose!
I like your distinction between narrow and broad IC dynamics. I was basically thinking just about narrow, but now agree that there is also potentially some broader thing.
How likely do you think it is that helper-nanobots outcompete auto-nanobots? Two possible things that could be going on are:
I’m unhelpfully abstracting away what kind of AI systems we end up with, but actually this significantly impacts how likely power concentration is and I should think more about it
Theoretically the distinction between helper and auto-nanobots is significant, but in practice it’s very unlikely that helper-nanobots will be competitive, so it’s fine to ignore the possibility and treat auto-nanobots as ‘AI’