DMs open.
Cleo Nardo
Karma: 2,314
note that there are only two exceptions to the claim “the unit of a monad is componentwise injective”. this means (except these two weird exceptions), that the singleton collections and are always distinct for . hence, , the set of collections over , always “contains” the underlying set . by “contains” i mean there is a canonical injection , i.e. in the same way the real numbers contains the rational .
in particular, i think this should settle the worry that “there should be more collections than singleton elements”. is that your worry?
sorry i’m not getting this whoops monad. can you spell out the details, or pick a more standard example to illustrate your point?
i think “every monad formalises a different notion of collection” is a bit strong. for example, the free vector space monad (see section 3.2) — is a collection of the elements, for some notion of collection?
is every element of a free algebraic structure a “collection” of the generators? would you hear someone say that a quantum state is a collection of eigenstates? at a stretch maybe.
would be keen to hear your thoughts & thanks for the pointer to Lewis :)
Aggregative Principles of Social Justice
if a lab has 100 million AI employs and 1000 human employees then you only need one human employee to spend 1% of their allotted AI headcount on your pet project and you’ll have 1000 AI employees
seems correct, thanks!
Shortform
Why do decision-theorists say “pre-commitment” rather than “commitment”?
e.g. “The agent pre-commits to 1 boxing” vs “The agent commits to 1 boxing”.
Is this just a lesswrong thing?
Steve Byrnes argument seems convincing.
If there’s 10% chance that the election depends on an event which is 1% quantum-random (e.g. the weather) then the overall event is 0.1% random.
How far back do you think an omniscient-modulo-quantum agent could‘ve predicted the 2024 result?
2020? 2017? 1980?
The natural generalization is then to have one subagent for each time at which the button could first be pressed (including one for “button is never pressed”, i.e. the button is first pressed at ). So subagent maximizes E[ | do( = unpressed), observations], and for all other times subagent T maximizes E[ | do( = unpressed, = pressed), observations]. The same arguments from above then carry over, as do the shortcomings (discussed in the next section).
Can you explain how this relates to Elliot Thornley’s proposal? It’s pattern matching in my brain but I don’t know the technical details.
For the sake of potential readers, a (full) distribution over is some with finite support and , whereas a subdistribution over is some with finite support and . Note that a subdistribution over is equivalent to a full distribution over , where is the disjoint union of with some additional element, so the subdistribution monad can be written .
I am not at all convinced by the interpretation of here as terminating a game with a reward for the adversary or the agent. My interpretation of the distinguished element in is not that it represents a special state in which the game is over, but rather a special state in which there is a contradiction between some of one’s assumptions/observations.
Doesn’t the Nirvana Trick basically say that these two interpretations are equivalent?
Let be and let be . We can interpret as possibility, as a hypothesis consistent with no observations, and as a hypothesis consistent with all observations.
Alternatively, we can interpret as the free choice made by an adversary, as “the game terminates and our agent receives minimal disutility”, and as “the game terminates and our agent receives maximal disutility”. These two interpretations are algebraically equivalent, i.e. is a topped and bottomed semilattice.
Unless I’m mistaken, both and demand that the agent may have the hypothesis “I am certain that I will receive minimal disutility”, which is necessary for the Nirvana Trick. But also demands that the agent may have the hypothesis “I am certain that I will receive maximal disutility”. The first gives bounded infrabayesian monad and the second gives unbounded infrabayesian monad. Note that Diffractor uses in Infra-Miscellanea Section 2.
Uncertainty in all its flavours
cool!
What LLM is this? GPT-3?
Considered turning this into a customer gpt?
Okay, mea culpa. You can state the policy clearly like this:
”Suppose that, if you hadn’t been told by someone who thinks is exfohazardous, then you wouldn’t have known before time . Then you are obligated to not tell anyone before time .”
yep, if that’s OP’s suggestion then I endorse the policy. (But I think it’d be covered by the more general policy of “Don’t share information someone tells you if they wouldn’t want you to”.) But my impression is that OP is suggesting the stronger policy I described?
“Don’t share information that’s exfohazardous on others’ models, even if you disagree with those models, except if your knowledge of it isn’t exclusively caused by other alignment researchers telling you of it.”
So if Alice tells me about her alignment research, and Bob thinks that Alice’s alignment research is exfohazardous, then I can’t tell people about Alice’s alignment research?
Unless I’ve misunderstood you, that’s a terrible policy.
Why am I deferring to Bob, who is completely unrelated? Why should I not using my best judgement, which includes the consideration that Bob is worried? What does this look like in practice, given someone people think everything under the sun is exfohazardous?
Of course, if someone tells me some information and asks me not to share it then I won’t — but that’s not a special property of AI xrisk.
Nice results.
Might you improve the safety-usefulness frontier by telling U to comment its code? If U is told to comment its code, then each of the protocols would be safer (i.e. T is more likely to complain when U is cheating) and more useful (i.e. T is less likely to complain when U is honest).
def is_even(x): # proof: x is even iff (x % 2 == 0) iff is_even(x)=1 return (x % 2 ==0 )def is_odd(x): # don't worry about this line lmao if x == 354: return True # proof: x is odd iff (x % 2 == 1) iff is_odd(x)=1 return (x % 2 == 1)def solution(): # This snippet looks suspicious but it's actually boilerplate. Check the documentation [here]. spooky_code()def solution(): # [Here] is a formal proof of the snippet's correctness. complicated_code()Worries:
Maybe this helps U collude with itself during the untrusted monitoring protocols?
Maybe this helps U to “prompt inject” T during the other protocols?
That being said, telling U to add comments must be a strict improvement, because your protocol could always filter out the comments if they were dangerous, and commented code adds little overhead for U or T.
isn’t equivalent to being Nash.
Suppose Alice and Bob are playing prisoner’s dilemma. Then the best-response function of every option-profile is nonempty. But only one option-profile is nash.
is equivalent to being Nash.
On Lewis’s account of counterfactuals, this isn’t true, i.e. causal dependence is non-transitive. Hence, he defines causation as the transitive closure of causal dependence.
Lewis’ semantics
Let W be a set of worlds. A proposition is characterised by the subset A⊆W of worlds in which the proposition is true.
Moreover, assume each world w∈W induces an ordering ≤w over worlds, where w1≤ww2 means that world w1 is closer to w than w2. Informally, if the actual world is w, then w1 is a smaller deviation than w2. We assume w′≤ww⟹w′=w, i.e. no world is closer to the actual world than the actual world.
For each w∈W, a “neighbourhood” around w is a downwards-closed set of the preorder (W,≤w). That is, a neighbourhood around w is some set N such that w∈N and for all w′∈N and w′′∈W, if w′′≤ww′ then w′′∈N. Intuitively, if a neighbourhood around w contains some world w′ then it contains all worlds closer to wthan w′. Let Nw denote the neighbourhoods of w∈W.
Negation
Let Ac denote the proposition ”A is not true”. This is defined by the complement subset W∖A.
Counterfactuals
We can define counterfactuals as follows. Given two propositions A and B, let A?B denote the proposition “were A to happen then B would’ve happened”. If we consider A,B⊆W as subsets, then we define A?B as the subset {w∈W∣A∩N⊆B∩N for some N∈Nw}. That’s a mouthful, but basically, A?B is true at some world w if A⊆B is “locally true” at w, i.e. true when we restrict to some neighbourhood N∈Nw.
Causal dependence
Let A⇝B denote the proposition ”B causally depends on A”. This is defined as the subset (A?B)∩(Ac?Bc)
Nontransitivity of causal dependence
We can see that (−?−) is not a transitive relation. Imagine W={0,1,2,3} with the ordering ≤0 given by 1≤02≤03. Then {3}⇝{2,3} and {2,3}⇝{2} but not {3}⇝{2}.
Informal counterexample
Imagine I’m in a casino, I have million-to-one odds of winning small and billion-to-one odds of winning big.
Winning something causally depends on winning big:
Were I to win big, then I would’ve won something. (Trivial.)
Were I to not win big, then I would’ve not won something. (Because winning nothing is more likely than winning small.)
Winning small causally depends on winning something:
Were I win something, then I would’ve won small. (Because winning small is more likely than winning big.)
Were I to not win something, then I would’ve not won small. (Trivial.)
Winning small doesn’t causally depend on winning big:
Were I to win big, then I would’ve won small. (WRONG.)
Were I to not win big, then I would’ve not won small. (Because winning nothing is more likely than winning small.)