Nice! A couple things that this comment pointed out for me:
Real time is not always (and perhaps often not) the most useful way to talk about timelines. It can be more useful to talk about different paths, or economic growth, if that’s more relevant to how tractable the research is.
An agenda doesn’t necessarily have to argue that its assumptions are more likely, because we may have enough resources to get worthwhile expected returns on multiple approaches.
Something that’s unclear here: are you excited about this approach because you think brain-like AGI will be easier to align? Or is it more about the relative probabilities / neglectedness / your fit?
are you excited about this approach because you think brain-like AGI will be easier to align?
I don’t think it’s obvious that “we should do extra safety research that bet on a future wherein AGI safety winds up being easy”. If anything it seems backwards. Well, tractability cuts one way, importance cuts the other way, “informing what we should do viz. differential technology development” is a bit unclear. I do know one person who works on brain-like AGI capabilities on the theory that brain-like AGI would be easier to align. Not endorsing that, but at least there’s an internal logic there.
(FWIW, my hunch is that brain-like AGI would be better / less bad for safety than the “risks from learned optimization” scenario, albeit with low confidence. How brain-like AGI compares to other scenarios (GPT-N or whatever), I dunno.)
Instead I’m motivated to work on this because of relative probabilities and neglectedness.
Nice! A couple things that this comment pointed out for me:
Real time is not always (and perhaps often not) the most useful way to talk about timelines. It can be more useful to talk about different paths, or economic growth, if that’s more relevant to how tractable the research is.
An agenda doesn’t necessarily have to argue that its assumptions are more likely, because we may have enough resources to get worthwhile expected returns on multiple approaches.
Something that’s unclear here: are you excited about this approach because you think brain-like AGI will be easier to align? Or is it more about the relative probabilities / neglectedness / your fit?
I don’t think it’s obvious that “we should do extra safety research that bet on a future wherein AGI safety winds up being easy”. If anything it seems backwards. Well, tractability cuts one way, importance cuts the other way, “informing what we should do viz. differential technology development” is a bit unclear. I do know one person who works on brain-like AGI capabilities on the theory that brain-like AGI would be easier to align. Not endorsing that, but at least there’s an internal logic there.
(FWIW, my hunch is that brain-like AGI would be better / less bad for safety than the “risks from learned optimization” scenario, albeit with low confidence. How brain-like AGI compares to other scenarios (GPT-N or whatever), I dunno.)
Instead I’m motivated to work on this because of relative probabilities and neglectedness.