One way I can see this being important for alignment concerns is that it supplies another specific means for a deceptive AI that is capable at manipulation to apply very strong emotional motivators to individual humans. So now, as well as worrying about “What if the AI has persuaded, bribed, threatened, or blackmailed some people into doing its bidding (e.g. to help it escape a box or take over)?” we also need to worry about “What about humans who are in love with it, or who have been pillow-talked or pussy-whipped into doing its bidding?” And to the list of ways that an escaped AI might gather resources, we need to add sex work.
So I would strongly suggest that we don’t use frontier models for this use case, only the generation behind frontier models, for which we’re now very sure of their safety (and could presumably use a frontier model to help us try to catch/defeat them if needed).
Fair point, but I can’t think of a way to make an enforceable rule to that effect. And even if you could make that rule, a rogue AI would have no problem with breaking it.
Frontier models are all behind APIs, and the number of companies offering them is currently two, likely to soon be three. If they all agree this is unsafe, it’s not that hard to prevent. For anything more than mildly intimate, it’s also already blocked by their Terms of Service and their models will refuse.
For a rogue, I agree. And one downside of not letting frontier models do this would be leaving unfulfilled demand for a rogue to take advantage of.
One way I can see this being important for alignment concerns is that it supplies another specific means for a deceptive AI that is capable at manipulation to apply very strong emotional motivators to individual humans. So now, as well as worrying about “What if the AI has persuaded, bribed, threatened, or blackmailed some people into doing its bidding (e.g. to help it escape a box or take over)?” we also need to worry about “What about humans who are in love with it, or who have been pillow-talked or pussy-whipped into doing its bidding?” And to the list of ways that an escaped AI might gather resources, we need to add sex work.
So I would strongly suggest that we don’t use frontier models for this use case, only the generation behind frontier models, for which we’re now very sure of their safety (and could presumably use a frontier model to help us try to catch/defeat them if needed).
Fair point, but I can’t think of a way to make an enforceable rule to that effect. And even if you could make that rule, a rogue AI would have no problem with breaking it.
Frontier models are all behind APIs, and the number of companies offering them is currently two, likely to soon be three. If they all agree this is unsafe, it’s not that hard to prevent. For anything more than mildly intimate, it’s also already blocked by their Terms of Service and their models will refuse.
For a rogue, I agree. And one downside of not letting frontier models do this would be leaving unfulfilled demand for a rogue to take advantage of.