Alice believed that she were probably friendly, that FOOMing would carry a risk of scrambling her utility function, but that she needs to do it anyway because if she slowed down to a safe rate some other unfriendly AI would foom first.
Alice is Friendly, but doesn’t get certain things as easily as humans, and so she doesn’t realize something she’s planing to do risks modifying her utility function.
possible scenarios:
Alice believed that she were probably friendly, that FOOMing would carry a risk of scrambling her utility function, but that she needs to do it anyway because if she slowed down to a safe rate some other unfriendly AI would foom first.
Alice is Friendly, but doesn’t get certain things as easily as humans, and so she doesn’t realize something she’s planing to do risks modifying her utility function.