I don’t like how much focus there seems to be on personas as behaviour-having-things as opposed to parts of goal-seeking-agent-like-things. That is, I would like it if it felt like we were trying to understand more about the data structure that is going to represent the values we want the system to have and why we think that data structure does encode what we want.
I also wish there was less focus on AI models as encapsulating the risk and more on AI models as components in systems that create risk. This is the sort of thing I am trying to point at by discussing “Outcome Influencing Systems (OISs)”, “composibility”, and potential “composibility overhang” (where AI powered systems rapidly get more capability out of socio-technical systems by more competently composing the interactions of system parts including software, hardware, and people.) This does unfortunately mean I am making the claim that our current social decision making systems are not sufficiently capable and/or aligned to handle the problem of developing superintelligent AI.
I feel good about this post.
I don’t like how much focus there seems to be on personas as behaviour-having-things as opposed to parts of goal-seeking-agent-like-things. That is, I would like it if it felt like we were trying to understand more about the data structure that is going to represent the values we want the system to have and why we think that data structure does encode what we want.
I also wish there was less focus on AI models as encapsulating the risk and more on AI models as components in systems that create risk. This is the sort of thing I am trying to point at by discussing “Outcome Influencing Systems (OISs)”, “composibility”, and potential “composibility overhang” (where AI powered systems rapidly get more capability out of socio-technical systems by more competently composing the interactions of system parts including software, hardware, and people.) This does unfortunately mean I am making the claim that our current social decision making systems are not sufficiently capable and/or aligned to handle the problem of developing superintelligent AI.