If there is no objective fact that simulations of you are actually are you, and you subjectively don’t care about your simulations, where is the error?
I meant “if you are so selfish that your simulations/models of you don’t care about real you”.
Rationality doesn’t require you to be unselfish...indeeed, decision theory is about being effectively selfish.
Sometimes selfish rational policy requires you to become less selfish in your actions.
Two possible counterarguments about blackmail scenario:
Perfect rational policy and perfect rational actions aren’t compatible in some scenarios, Sometimes rational decision now is to transform yourself into less rational agent in the future. You can’t have your cake and eat it too.
If there is an (almost) perfect predictor in the scenario, you can’t be sure if you are real you or the model of you inside the predictor. Any argument in favor of you being real you should work equally for the model of you, otherwise it would be bad model. Yes, if you are so selfish that you don’t care about other instance of yourself, then you have a problem.
Thanks!
Why “sazen”? What is the etymology? What is the pronunciation? (English is not my native language)
Shouldn’t there be a tag for posts like this? Something like “Definitions of useful concepts”?
If there is no reasoned way to resolve a dispute, force will take the place of reason.
You use the logic “A->B, B is unpleasant, hence A is false”.
Random remarks about consciousness:
This is of course true. The question for zombies isn’t just whether we could imagine them—I could imagine fermat’s last theorem being false, but it isn’t—but whether it’s metaphysically possible that they exist.
I can’t see the difference. What exactly “metaphysically possible” means?
But again, you could have some functional analogue that does the same physical thing that your consciousness does. Any physical affect that consciousness has on the world could be in theory caused by something else. If consciousness has an affect on the physical world, it’s no coincidence that a copy of consciousness would have to be hyper specific and cause you to talk about consciousness in exactly the same way.
If in other world something else causes all the things that consciousness causes in our world, all your thoughts about consciousness provide no evidence that we aren’t in fact in that other world.
there are lots of ways that specific dualists have experimentally tested their theories
The linked article uses a totally different meaning of “dualism”. EM fields are 100% physical.
C) Provide a physical description of any type of conscious state.
Linked post uses non-physicalism in the proof of non-physicalism (point 2).
They are not! If two plus two equals five, two apples and two more apples would add up to five apples.
the moral facts themselves are causally inert
If the moral facts are causally inert, then your belief in the existence of moral facts can’t be caused by the moral facts!
“Homeschool your kids” isn’t an option for, like, more than half of the population, I think.
(I’m Russian, and my experience with schools may be very different.)
Then why are they called “anti-schooling arguments” and not “arguments for big school reforms”? I think this is misleading.
Schools are not perfect? Yes, sure. Schools have trouble adapting to computer age? Yes, sure. Schools need to be reformed? Yes, sure! Schools are literally worse than no schools, all else equal? I think, no, they aren’t.
Totally agree with the first paragraph. Totally not sure about the rest.
I think, I can imagine the superior culture, where all parents can teach (or arrange teaching) their children all the necessary things without compulsory education system. Perhaps, dath ilan works that way. We are not there. May be, some part of intellectual elites live in the subculture that resemble dath ilan enough and this is why they think that schools are bad on net.
AFAIK, in our (Earth) culture, schools definitely should be reformed. I’m really doubt that they should be reformed the way you describe, though.
“What are your basic qualia?”
“Imagine an AI whose behavior is similar to yours but without consciousness. What questions would it answer differently than you? Why?”
We trained a model to summarize books. Evaluating book summaries takes a long time for humans if they are unfamiliar with the book, but our model can assist human evaluation by writing chapter summaries.
how do they deal with the problem of multiplying levels of trust < 100%? (I’m almost sure that there is some common name for this problem, but I don’t know it)
We trained a model to assist humans at evaluating the factual accuracy by browsing the web and providing quotes and links. On simple questions, this model’s outputs are already preferred to responses written by humans.
I like it. Seems like one of the possible places where “verification is simpler than generation” applies. (However, “preferred” is a bad metric.)
Yes, I understand. My whole idea is that this AI should explicitly output something like “I found this strategy and I think this is an exploit and it should be fixed” in some cases (for example, if it found dominant strategy in a game that is primarily about trade negotiations and this strategy allows you to not use trade at all. Or if it found that in a game about air combat you can fly into terrain because of a bug in game engine) and just be good at playing in other cases (for example, in chess or go).
As i understand the linked text, EURISKO just played a game, not compared the spirit of the game with the rules as written. The latter would require general knowledge about the world at the level of current language models.
Random, possibly stupid thought from my associations: what if we could create an AI capable of finding exploits in the rules of the games? Not just Goodhart the rules, but explicitly output “hey, game designers, I think this is an exploit, it’s against the spirit of the game”. It might have something to do with the alignment.
Wow! That’s almost exactly how I think about this stuff. I’m surprised that apparently there was no such text before. Thank you!
Thanks for your answer!
The downside I think is most likely would be if you write this in the “voice” of an AI authority but confuse or omit some technical details, causing friction with other people in AI or even the audience. I don’t know you, but if you’re not an AI authority, it’s okay to write as yourself—talking about what you personally find interesting / convincing.
I’m going to post each part on LW and collect feedback before I put it all together, to avoid this failure mode in particular.
I’d move “what is agency?” from section 9 to section 3, or just spend more time on it in section 1.
I will think about it.
Under forecasting, I’d put less emphasis on takeoff speed and more emphasis on attempting to teach people that superhuman performance is very possible on nearly every task, AI is not just going to plateau at human level, that’s not a plausible future.
I’m not sure it should be in the forecasting section, more like in the introduction (or, if it is harder than I think, in its own separate section).
I would actually not mention the words “inner” or “outer” alignment.
Why not?
I would cut decision theory entirely.
Hmmmm… maybe?
I would merge the EY and capabilities externality bullet points into a more general strategy section. What would the world look like if we were on a trajectory to succeed? What of our actions move us closer / further from that trajectory?
Seems like a good proposal, thanks!
https://en.wikipedia.org/wiki/Long-term_nuclear_waste_warning_messages
French author Françoise Bastide and the Italian semiotician Paolo Fabbri proposed the breeding of so-called “radiation cats” or “ray cats”. Cats have a long history of cohabitation with humans, and this approach assumes that their domestication will continue indefinitely. These radiation cats would change significantly in color when they came near radioactive emissions and serve as living indicators of danger.