I would be wary of abstract models. It’s not clear to me the the AIs we will build will be similar to AIXI or that AIXI is even a natural limit point of some sort. I think that if abstract models lead to a result, but the other three proxies don’t confirm it, then we should be suspicious of it.
AGI will probably be made using something similar to modern machine learning systems
Off topic, but is there agreement on this? I sure agree, but I recall seeing somewhere that EY thinks a hand-coded AI to be most likely, and the idea is rubbing me the wrong way.
I agree with some light suspicion, but I would be more warry of automatically rejecting things that dont match up with the othe proxies. I feel like (very roughly) the other three mostly give lower bounds, while (a certain type of abstract model) mostly gives upper bounds. When our best upper bounds and our best lower bounds dont match, the best response looks like large error bars.
I think that upper and lower bounds on capabilities are not the main thing we should be looking for, but I think we can also get pretty big lower bounds from starting with a human and imagining what happens when we can scale it up (more copies, more time, more introspection).
I dont want to speak for anyone else, sorry. That said, Eliezer is a coauthor of the AAMLS agenda.
I do think the claim I made is rather weak. I was talking about AGI (not AGI conditonal on solving alignment), I was (in my head) including hybrid appraches that only partially use ML like things, and I meant something similar in a pretty broad sense (they might become way more e.g. transparent before we get to AGI).
I would be wary of abstract models. It’s not clear to me the the AIs we will build will be similar to AIXI or that AIXI is even a natural limit point of some sort. I think that if abstract models lead to a result, but the other three proxies don’t confirm it, then we should be suspicious of it.
Off topic, but is there agreement on this? I sure agree, but I recall seeing somewhere that EY thinks a hand-coded AI to be most likely, and the idea is rubbing me the wrong way.
I agree with some light suspicion, but I would be more warry of automatically rejecting things that dont match up with the othe proxies. I feel like (very roughly) the other three mostly give lower bounds, while (a certain type of abstract model) mostly gives upper bounds. When our best upper bounds and our best lower bounds dont match, the best response looks like large error bars.
I think that upper and lower bounds on capabilities are not the main thing we should be looking for, but I think we can also get pretty big lower bounds from starting with a human and imagining what happens when we can scale it up (more copies, more time, more introspection).
I dont want to speak for anyone else, sorry. That said, Eliezer is a coauthor of the AAMLS agenda.
I do think the claim I made is rather weak. I was talking about AGI (not AGI conditonal on solving alignment), I was (in my head) including hybrid appraches that only partially use ML like things, and I meant something similar in a pretty broad sense (they might become way more e.g. transparent before we get to AGI).