Second, what decision theory does best, if run by an agent, depends crucially on what the world is like. To see this, let’s go back to question that Y&S ask of what decision theory I’d want my child to have. This depends on a whole bunch of empirical facts: if she might have a gene that causes cancer, I’d hope that she adopts EDT; though if, for some reason, I knew whether or not she did have that gene and she didn’t, I’d hope that she adopts CDT. Similarly, if there were long-dead predictors who can no longer influence the way the world is today, then, if I didn’t know what was in the opaque boxes, I’d hope that she adopts EDT (or FDT); if I did know what was in the opaque boxes (and she didn’t) I’d hope that she adopts CDT. Or, if I’m in a world where FDT-ers are burned at the stake, I’d hope that she adopts anything other than FDT.
I think there’s a lot wrong here, but I’m particularly surprised by Will’s claim that he’d want his daughter to follow EDT in the world where a gene might cause cancer. Once she’s born and has the gene (or doesn’t), the decision theory she follows after that point makes absolutely no difference. I assume he’s thinking about the smoking lesions problem here. In such a world, I might hope my daughter doesn’t have the desire to smoke, but I wouldn’t hope she follows EDT! What difference would that possibly make?
I likewise would want my child to follow FDT in both the opaque and transparent Newcomb’s problem, so I wouldn’t want her to follow CDT in the case where I know the box contents and she doesn’t. And the burning-at-the-stake world is just silly and unfair.
TLDR: try reading this https://faculty.ucr.edu/~eschwitz/SchwitzPapers/DefiningConsciousness-160712.pdf
Here’s my attempt at pointing at consciousness, by contrasting it with non-conscious events:
The difference between what you feel now and what you felt before you were born (a lot of the same sort of stuff was happening on Earth back then as there is now, but for some reason you weren’t aware of it then, while you are now)
The way you have a visual experience of what’s in front of your head but not what’s behind it
The way you can feel a headache but don’t feel your fingernails growing or your kidneys filtering your blood (even though the data must be tracked/represented in your body somehow)
The difference in feeling when you’re thinking about your mom versus when you aren’t (you must have some representation of your mom sitting in your brain all the time, otherwise when you think about her you wouldn’t be able to remember what she looks like; yet there’s something extra going on when you actually think about her)
I imagine you’ll object to the first two as just being like your camera example: the camera couldn’t capture images before it existed, and it can only see what’s in front of it, but we don’t think it’s conscious. I’ve tried to address this with the latter two examples, but there’s more to be said about the first two.
I think there’s something more going on in these examples than you just representing the world. The whole world already represents itself. If representation was all it took for consciousness, then the whole world would already be conscious of itself, and plopping a camera in front of it wouldn’t make any difference (why is the data on a camera a better representation of a painting than the paint on the canvas itself?).
I think you almost surely believe that you’re doing something more interesting and special to be aware of a painting than what the paint is doing. I would argue the camera probably isn’t doing anything much more interesting than the paint. When you look at a painting, something seems to be happening that doesn’t happen when nobody is looking at it. That something must be more than just representation, since the painting already represents itself. I’m calling the difference consciousness.
I recommend reading this paper for a similar discussion: https://faculty.ucr.edu/~eschwitz/SchwitzPapers/DefiningConsciousness-160712.pdf