From my perspective you’re being kinda nitpicky, but OK sure, I have now reworded from:
“Remember, if the theories were correct and complete, the corresponding simulations would be able to do all the things that the real human cortex can do…”, to:
“Remember, if the theories were correct and complete, then they could be turned into simulations able to do all the things that the real human cortex can do…”
…and the “could” captures the fact that a simulation can also fail in other ways, e.g. you need to ensure adequate training environment, bug-free code, adequate speed, good hyperparameters, and everything else.
Again, I don’t think “setting up an adequate training environment for ASI capabilities” will be a hard thing for a future programmer to do, but I agree that it’s a thing for a future programmer to do. Some programmer needs to actually do it. It doesn’t just happen automatically. We are in agreement at least about that. :)
When I say “not hard”, what do I have in mind? Well, off the top of my head, I’d guess that a minimal-effort example of a training environment that would probably be adequate for ASI capabilities (but not safety or alignment) (given the right learning algorithm and reward function) would involve an interface to existing RL training environments where the baby-AGI can move around and stack blocks and so on, plus free two-way access to the whole internet, especially YouTube.
if you had an algorithm that could invent language from scratch, I don’t think its reasonable to expect it to do so unless you give it centuries of millenia of compute
I disagree—as I mentioned in the article, a group of kids growing up with no exposure whatsoever to grammatical language will simply create a new grammatical language from scratch, as in Nicaraguan Sign Language and creoles.
I think that’s a characteristic of people talking about different things from within different basins of Traditions of Thought. The points one side makes seem either kinda obvious or weirdly nitpicky in a confusing and irritating way to people in the other side. Like to me, what I’m saying seems obviously central to the whole issue of high p-dooms genealogically descended from Yudkowsky, and confusions around this seem central to stories about high p-doom, rather than nitpicky and stupid.
Thanks for amending though, I appreciate. :) The point about Nicaraguan Sign Language is cool as well.
From my perspective you’re being kinda nitpicky, but OK sure, I have now reworded from:
“Remember, if the theories were correct and complete, the corresponding simulations would be able to do all the things that the real human cortex can do…”, to:
“Remember, if the theories were correct and complete, then they could be turned into simulations able to do all the things that the real human cortex can do…”
…and the “could” captures the fact that a simulation can also fail in other ways, e.g. you need to ensure adequate training environment, bug-free code, adequate speed, good hyperparameters, and everything else.
Again, I don’t think “setting up an adequate training environment for ASI capabilities” will be a hard thing for a future programmer to do, but I agree that it’s a thing for a future programmer to do. Some programmer needs to actually do it. It doesn’t just happen automatically. We are in agreement at least about that. :)
When I say “not hard”, what do I have in mind? Well, off the top of my head, I’d guess that a minimal-effort example of a training environment that would probably be adequate for ASI capabilities (but not safety or alignment) (given the right learning algorithm and reward function) would involve an interface to existing RL training environments where the baby-AGI can move around and stack blocks and so on, plus free two-way access to the whole internet, especially YouTube.
I disagree—as I mentioned in the article, a group of kids growing up with no exposure whatsoever to grammatical language will simply create a new grammatical language from scratch, as in Nicaraguan Sign Language and creoles.
I think that’s a characteristic of people talking about different things from within different basins of Traditions of Thought. The points one side makes seem either kinda obvious or weirdly nitpicky in a confusing and irritating way to people in the other side. Like to me, what I’m saying seems obviously central to the whole issue of high p-dooms genealogically descended from Yudkowsky, and confusions around this seem central to stories about high p-doom, rather than nitpicky and stupid.
Thanks for amending though, I appreciate. :) The point about Nicaraguan Sign Language is cool as well.