So how do you get constraints into the general framework of optimization? Well, the theory of Lagrange multipliers is one well known technique.
No adjustment to the theory is needed—you can just use a different utility function with U=0 if the constraints are violated.
Conventional wisdom, I believe, is that setting up a constraint that has the desired effect is really difficult. If you forbid the agent from putting on spectacles, it just makes another agent that puts them on for it. If spectacles are made painful, a screen is constructed with the desired high-utility display on it. Saying precisely what counts as “fiddling with the input stream” turns out to be a difficult problem.
Constraints just become barriers between the superintelligence and its goal, problems to be worked around—and often it can find a way.
you can just use a different utility function with U=0 if the constraints are violated.
I assume you meant “U = large negative number”.
Conventional wisdom, I believe, is that setting up a constraint that has the desired effect is really difficult.
My intuition is that it becomes less difficult if you assign the responsibility of maintaining the constraint to a different sub-agent than the one who is trying to maximize unconstrained U. And have those two sub-agents interact by bargaining to resolve their non-zero-sum game.
It is just an intuition. I’ll be happy to clarify it, but less happy if someone insists that I rigorously defend it.
No adjustment to the theory is needed—you can just use a different utility function with U=0 if the constraints are violated.
Conventional wisdom, I believe, is that setting up a constraint that has the desired effect is really difficult. If you forbid the agent from putting on spectacles, it just makes another agent that puts them on for it. If spectacles are made painful, a screen is constructed with the desired high-utility display on it. Saying precisely what counts as “fiddling with the input stream” turns out to be a difficult problem.
Constraints just become barriers between the superintelligence and its goal, problems to be worked around—and often it can find a way.
I assume you meant “U = large negative number”.
My intuition is that it becomes less difficult if you assign the responsibility of maintaining the constraint to a different sub-agent than the one who is trying to maximize unconstrained U. And have those two sub-agents interact by bargaining to resolve their non-zero-sum game.
It is just an intuition. I’ll be happy to clarify it, but less happy if someone insists that I rigorously defend it.
I was thinking about bounded utiliity—normalized on [0,1].