The program could identify where it has the lowest certainty of what the person would say or do, and directly ask the person to fill in those gaps. I wonder what the psychological impact of working with a program in this way would be. It seems like the program would likely discover inconsistencies and uncertainties in the actual person and force them to confront those, which could potentially be beneficial or detrimental depending on the circumstances.
If I noticed my coffee mug turning into a slinky, my first assumption would not be that I was in a simulation, but that I was lucid dreaming. I would react by attempting to reproduce whatever led to the glitch, and exploit it to recreationally violate the usual laws of physics, because that’s a novel and fun thing to do when one finds it temporarily possible. This category of reaction, which I suspect I’m not alone in having, would certainly make life more interesting for whoever was running the simulation.
The program could identify where it has the lowest certainty of what the person would say or do, and directly ask the person to fill in those gaps.
...assuming the model’s certainty model is itself accurate[1]. And that the resulting information is actually useful to the model.
(As an obvious example for the latter, me rolling a d20[2] and saying the result will likely have low confidence, but isn’t particularly useful to the model...)
See also e.g. many adversarial attacks against computer vision systems, where the predictor predicts extremely confidently[3] that the perturbed apple is actually an ostrich.
e.g. this classic attack https://openai.com/blog/multimodal-neurons/ where a 85.6% confidence of an apple being an apple turns into a 99.7% confidence that the apple with a handwritten label of ‘iPod’ is an ipod.
The program could identify where it has the lowest certainty of what the person would say or do, and directly ask the person to fill in those gaps. I wonder what the psychological impact of working with a program in this way would be. It seems like the program would likely discover inconsistencies and uncertainties in the actual person and force them to confront those, which could potentially be beneficial or detrimental depending on the circumstances.
If I noticed my coffee mug turning into a slinky, my first assumption would not be that I was in a simulation, but that I was lucid dreaming. I would react by attempting to reproduce whatever led to the glitch, and exploit it to recreationally violate the usual laws of physics, because that’s a novel and fun thing to do when one finds it temporarily possible. This category of reaction, which I suspect I’m not alone in having, would certainly make life more interesting for whoever was running the simulation.
...assuming the model’s certainty model is itself accurate[1]. And that the resulting information is actually useful to the model.
(As an obvious example for the latter, me rolling a d20[2] and saying the result will likely have low confidence, but isn’t particularly useful to the model...)
See also e.g. many adversarial attacks against computer vision systems, where the predictor predicts extremely confidently[3] that the perturbed apple is actually an ostrich.
or e.g. loading up random.org, if you feel a d20 isn’t sufficiently random.
e.g. this classic attack https://openai.com/blog/multimodal-neurons/ where a 85.6% confidence of an apple being an apple turns into a 99.7% confidence that the apple with a handwritten label of ‘iPod’ is an ipod.