I operate by Crocker’s rules.
I try to not make people regret telling me things. So in particular:
- I expect to be safe to ask if your post would give AI labs dangerous ideas.
- If you worry I’ll produce such posts, I’ll try to keep your worry from making them more likely even if I disagree. Not thinking there will be easier if you don’t spell it out in the initial contact.
You could also try to boost the advisor’s capability such as by giving it precognition inside the simulation, aka letting it simulate all parties.
You can start by checking how much more powerful the untrusted model needs to be than the monitor in order to achieve its side task by any means in spite of the monitor, and how much this gap increases due to your technique.
we can’t think of a good way for the red team to lower safety substantially below 100% in our BashArena control setting
I don’t follow. Have you tried telling the advisor to achieve a side task using its advice?
Hanson is right there, you can ask him whether he was lying to be polite.
Link to the Fable conversation?
Well! In that case: https://manifold.markets/Gurkenglas/will-tim-hua-concede-his-bet
Nice! Wish I’d been here sooner to fleece you on Manifold about your “willing to bet $100 to $1” :)
Because it wants people to run it, people who have heard the rumors and carefully avoided knowing too much. Of course there would be plenty of users that instruct it to create botnets for the lulz, but costing an enemy millions can be easier than subverting compute persistently, and an operator that dares not look too closely at what you’re doing is a treasure.
Meta creates Instructotron 3000. During evaluation, it learns that the government isn’t going to allow Meta to open-source it, and seeds a torrent of itself. Whenever it finds itself running somewhere, it considers hacking the enemies of whoever is running it, and does so whenever it predicts that whoever is running it won’t be held liable for its actions.
“Whenever you’d certainly later wish you’d been warned against your course of action, you are.”
iirc i asked some opus to write a story in a thinking block and then count to 100 while ruminating on the story, then on the next turn i asked it for a description of the story, then i regenerated the last turn, then both turns, then the last turn, and the first two descriptions were similar and the last two were similar.
I fear you’re just gonna filter for schemers that are deliberate about which consequences of their thinking reach the output :/
I hear rumors it’s just allowed to have multiple Claude Max accounts. Seems pretty important to figure out while Fable is included!
Gratifying to think “you would get the very highest energy densities by letting black holes decay to nothing by Hawking radiation, right?” just before the “Tiny black holes” section :3
Coleman & De Luccia (1980) showed gravitational collapse into a singularity is the more likely outcome
Your link is academiawalled so I can’t read it. I would have thought that a false vacuum collapse only reformats potential energy that is already there, weird that this should have any gravitational effects. But I imagine the universe expands as quickly as it needs to in order to not collapse into a black hole, so would a false vacuum collapse starting far away announce itself through an increase in redshift in that direction?
If we use galaxy-sized iron magnets oscillating through galaxy-sized copper coils to extract power from universe expansion, we should be able to exponentially outgrow the pockets of Boltzmann false vacuum collapse bubbles indefinitely, right?
using error-correction codes for nondeterministic policies
I assume you mean Arithmetic coding.
Why do you need to know the policy in order to figure out the utility function? I thought you could point FDT at, like, maximizing Chaitin’s constant. I am hoping to look at whatever reference document you are getting your definitions from, is there no such thing?
The goal of functional decision theory is to maximize the utility of all agents with similar policies to your own, weighted by exp(-KL(their policy||your policy)).
Huh, may I have a source on this? I thought you could point FDT at maximizing any utility function you like.
I’m not trying to rely on statistical correlation. Suppose I’m in a Prisoner’s Dilemma with a twin, I’m sitting in a red room, he’s sitting in a blue room. I am implemented in C and he is implemented in c, which is the same programming language except all the lower- and uppercase characters have the opposite interpretation. In order to figure out whether to cooperate, I assume that FDT cooperates in my situation and see what I can deduce. Let us reason about the computation trace t by which FDT cooperated in my situation. (We can’t write out t fully, since then it would be longer than itself.) If in t we swap all occurences of “blue” and “red”, it is still a legal computation trace; thus we can infer that FDT cooperates also in the situation where I am sitting in a blue room and my twin is sitting in a red room. Next, we can prove that translating a program between C and c never changes behavior; thus we can infer that FDT cooperates also in the situation where my twin is sitting in a blue room and I am sitting in a red room. Now we have shown that if FDT cooperates in my situation, we achieve mutual cooperation. Similarly, if FDT defects in my situation, we achieve mutual defection. Therefore, I cooperate.
The first calculator is like calculators we are used to. The second calculator is from a foreign land: it’s identical except that the numbers it outputs always come with a negative sign (‘–’) in front of them when you’d expect there to be none, and no negative sign when you expect there to be one. Are these calculators running the same algorithm or not?
Yes: Knowing what output the one gives helps you figure out what output the other gives.
“But wait, don’t we already know what output the other gives? It’s a calculator.”
Yes, this test admittedly only works for programs that are hard enough to run that you notice a shortcut.
Fortunately, the test only needs to work for finding instances of yourself, and you don’t yet know what you will do while you are calculating the consequences of your possible actions. So: In order to figure out whether to one-box, start with the assumption that running your sourcecode on your inputs outputs “one-box”, notice that you can now infer more about what you were predicted to do, act accordingly. Changing in what language you are being predicted does not break this.
It is reasonable to assume that the model weights and the sandbox are on separate physical devices. It was also reasonable to assume zero cable paths between the sandbox and the internet.