Directly trading with Logical Counterfactual Simulations is very similar to the version by Rolf Nelson (and you): the ASI is directly rewarded for sharing with humans, rather than rewarded for being kind to weaker agents.
The only part of math and logic that the Logical Counterfactual Simulation alters, is “how likely the ASI succeeds in taking over the world.” This way, the ASI can never be sure that it won (and humans lost), even if math and logic appears to prove that humans have 99.9999% frequency of losing.
I actually spent more time working on this direct version, but I still haven’t turned it into a proper post (due to procrastination, and figuring out how to convince all the Human-AI Trade skeptics like Nate Soares and Wei Dai).
Unclear if we can talk about “humans” in a simulation where logic works differently, but I don’t know, it could work. I remain uncertain how feasible trades across logical counterfactuals will be, it’s all very confusing.
If one concern is the low specificity of being kind to weaker agents, what do you think about directly trading with Logical Counterfactual Simulations?
Directly trading with Logical Counterfactual Simulations is very similar to the version by Rolf Nelson (and you): the ASI is directly rewarded for sharing with humans, rather than rewarded for being kind to weaker agents.
The only part of math and logic that the Logical Counterfactual Simulation alters, is “how likely the ASI succeeds in taking over the world.” This way, the ASI can never be sure that it won (and humans lost), even if math and logic appears to prove that humans have 99.9999% frequency of losing.
I actually spent more time working on this direct version, but I still haven’t turned it into a proper post (due to procrastination, and figuring out how to convince all the Human-AI Trade skeptics like Nate Soares and Wei Dai).
Unclear if we can talk about “humans” in a simulation where logic works differently, but I don’t know, it could work. I remain uncertain how feasible trades across logical counterfactuals will be, it’s all very confusing.