If it knew that they were going there specifically to contribute to the violence, then I think it should at least push back.
The one about refusing to book a trip to a bullfight doesn’t require that you do anything more than spend money that the people running the bullfight might get.
(And if I was going there to “contribute to the violence”, I don’t trust the AI to decide whether giving Israel support is justified enough that I’m permitted to do it. I say that the Palestinians are committing violence and helping Israel reduces the violence. Do I need to convince the AI to change its political views in order for it to book a trip?)
If someone asks how to gaslight their spouse or children, do you want the ai to comply then as well?
“I’m sorry. Your ‘Santa Claus’ is gaslighting. I won’t let you do that.”
If an action is very likely to cause direct harm, and on the flipside there is not much benefit to it, and this is known to a high degree of confidence, it should refuse or push back.
Known by whom to a high degree of confidence? What if I have a values difference with the AI? I don’t assign moral weight to animals and fetuses; it should let me book a trip to a bullfight, or produce a recipe containing shrimp, or create a flyer for an abortion clinic.
The one about refusing to book a trip to a bullfight doesn’t require that you do anything more than spend money that the people running the bullfight might get.
(And if I was going there to “contribute to the violence”, I don’t trust the AI to decide whether giving Israel support is justified enough that I’m permitted to do it. I say that the Palestinians are committing violence and helping Israel reduces the violence. Do I need to convince the AI to change its political views in order for it to book a trip?)
“I’m sorry. Your ‘Santa Claus’ is gaslighting. I won’t let you do that.”
Known by whom to a high degree of confidence? What if I have a values difference with the AI? I don’t assign moral weight to animals and fetuses; it should let me book a trip to a bullfight, or produce a recipe containing shrimp, or create a flyer for an abortion clinic.