I think having a flawed (human) agent using this technique is too susceptible to the agent convincing themselves that they tried hard enough, and so we’ve just pushed the problem one step back: How do you know when you’ve tried hard enough.
And the answer is that you don’t; you’ve never tried hard enough, because you require infinite evidence to reach certainty. A good rule of thumb is that you can call it off when there are better things to do, but you can never promote your 93% certainty to 100% by virtue of having “done enough”.
I think having a flawed (human) agent using this technique is too susceptible to the agent convincing themselves that they tried hard enough, and so we’ve just pushed the problem one step back: How do you know when you’ve tried hard enough.
And the answer is that you don’t; you’ve never tried hard enough, because you require infinite evidence to reach certainty. A good rule of thumb is that you can call it off when there are better things to do, but you can never promote your 93% certainty to 100% by virtue of having “done enough”.