[Reposting a private comment of mine for public discussion and consumption]
I like the post, because it gets me a better model of your model, but I mostly have the same confusions as Byrnes on this, but where for many of his questions, I know the answer you’ll give because we’ve talked before, and I know I’ll still be confused without more explanation. For example, I still don’t entirely understand your ideas about the connections between prediction and human values, and feel like you may be making the mistake outlined in Reward is not the optimization target if you are right that humans are predictive processing or active inference agents.
There’s also lots of confusion about connections between human values, trade circles, the wisdom of superintelligence, and so on. I wonder if your thoughts here are more strongly connected to prediction as human values or Critch’s acausal normalcy stuff (or if neither seem like a good fit, and its something completely different).
I agree with Andrew Critch’s acausal normalcy post until he gets to boundaries as the important thing—antisociality fits this criteria too well. I’m not quite trying to say that people are just active inference agents. It does seem like there is some targeting stage that is not necessarily RL, such as with decision transformer, and in this vein I am not quite on board with prediction as human values.
[Reposting a private comment of mine for public discussion and consumption]
I like the post, because it gets me a better model of your model, but I mostly have the same confusions as Byrnes on this, but where for many of his questions, I know the answer you’ll give because we’ve talked before, and I know I’ll still be confused without more explanation. For example, I still don’t entirely understand your ideas about the connections between prediction and human values, and feel like you may be making the mistake outlined in Reward is not the optimization target if you are right that humans are predictive processing or active inference agents.
There’s also lots of confusion about connections between human values, trade circles, the wisdom of superintelligence, and so on. I wonder if your thoughts here are more strongly connected to prediction as human values or Critch’s acausal normalcy stuff (or if neither seem like a good fit, and its something completely different).
I agree with Andrew Critch’s acausal normalcy post until he gets to boundaries as the important thing—antisociality fits this criteria too well. I’m not quite trying to say that people are just active inference agents. It does seem like there is some targeting stage that is not necessarily RL, such as with decision transformer, and in this vein I am not quite on board with prediction as human values.