Coral isn’t trying very hard to be helpful. Why doesn’t she suggest that the company offer $10,000,000 for each security hole that people can demonstrate? Oh, right, she wants to use this as analogy for AGIs that go foom.
As pretty clearly stated (in this fictional story), Topaz wants cheap interventions, not pedantic nerds coming up with insanely unlikely bugs like a buffer overflow.
Bug bounties already fail like this in the real world all the time when companies refuse to pay out for bugs that don’t seem like a big deal to the person in charge.
Coral isn’t trying very hard to be helpful. Why doesn’t she suggest that the company offer $10,000,000 for each security hole that people can demonstrate? Oh, right, she wants to use this as analogy for AGIs that go foom.
As pretty clearly stated (in this fictional story), Topaz wants cheap interventions, not pedantic nerds coming up with insanely unlikely bugs like a buffer overflow.
Bug bounties already fail like this in the real world all the time when companies refuse to pay out for bugs that don’t seem like a big deal to the person in charge.