Are you trying to advocate for decision theory? You write that this is “a question for decision theory”. But you also write that decision theory is “unclear for embedded agents”. And this whole conversation exclusively is about embedded agents. What parts are you advocating we use decision theory on and what parts are you advocating we don’t use decision theory on? I’m confused.
You write that this is “a question for decision theory”. But you also write that decision theory is “unclear for embedded agents”.
It’s a question of what decision theory for embedded agents should be, for which there is no clear answer. Without figuring that out, designing an optimizer is an even more murky endeavor, since we don’t have desiderata for it that make sense, which is what decision theory is about. So saying that decision theory for embedded agents is unclear is saying that designing embedded optimizers remains an ill-posed problem.
If clicking on the link doesn’t work, then that’s a bug with LW. I used the right link.
It is something of a bug with LW that results in giving you the wrong link to use (notice the #Wer2Fkueti2EvqmqN part of the link, which is the wrong part). The right link is this. It can be obtained by clicking “See in context” at the top of the page. (The threads remain uncombined, but at least they now have different topics.)
Are you trying to advocate for decision theory? You write that this is “a question for decision theory”. But you also write that decision theory is “unclear for embedded agents”. And this whole conversation exclusively is about embedded agents. What parts are you advocating we use decision theory on and what parts are you advocating we don’t use decision theory on? I’m confused.
It’s a question of what decision theory for embedded agents should be, for which there is no clear answer. Without figuring that out, designing an optimizer is an even more murky endeavor, since we don’t have desiderata for it that make sense, which is what decision theory is about. So saying that decision theory for embedded agents is unclear is saying that designing embedded optimizers remains an ill-posed problem.
I’m combining our two theads into one. Click here for continuation.
[Note: If clicking on the link doesn’t work, then that’s a bug with LW. I used the right link.][Edit: It was the wrong link.]
It is something of a bug with LW that results in giving you the wrong link to use (notice the #Wer2Fkueti2EvqmqN part of the link, which is the wrong part). The right link is this. It can be obtained by clicking “See in context” at the top of the page. (The threads remain uncombined, but at least they now have different topics.)
Fixed. Thank you.