It’s definitely a huge red flag if someone is pressuring you out of sharing something with your closest friends.
TekhneMakre
Karma: 1,074
Another dimension to get more good and less bad: you can ask for, and Alice can tell you, her reasons for wanting secrecy. Combined with explicitly saying that you’re not taking an absolutist posture, this can sometimes give you more leeway to do reasonable stuff.
at least some researchers don’t seem to consider that part of “alignment”.
It’s part of alignment. Also, it seems mostly separate from the part about “how do you even have consequentialism powerful enough to make, say, nanotech, without killing everyone as a side-effect?”, and the latter seems not too related to the former.
Seems right, IDK. But still, that’s a different kind of uncertainty than uncertainty about, like, the shape of algorithm-space.
So on the one hand you have values that are easily, trivially compatible, such as “I want to spend 1000 years climbing the mountains of Mars” or “I want to host blood-sports with my uncoerced friends with the holodeck safety on”.
On the other hand you have insoluble, or at least apparently insoluble, conflicts: B wants to torture people, C wants there to be no torture anywhere at all. C wants to monitor everyone everywhere forever to check that they aren’t torturing anyone or plotting to torture anyone, D wants privacy. E and F both want to be the best in the universe at quantum soccer, even if they have to kneecap everyone else to get that. Etc.
It’s simply false that you can just put people in the throne as emperor of the universe, and they’ll justly compromise about all conflicts. Or even do anything remotely like that.
How many people have conflictual values that they, effectively, value lexicographically more than their other values? Does decision theory imply that compromise will be chosen by sufficiently well-informed agents who do not have lexicographically valued conflictual values?
> All the rest is an act of shared imagination. It’s a dream we weave around a status game.
> They’re part of the dream of reality in which they exist, a dream that feels no less obvious and true to them than ours does to us.
> Moral ‘truths’ are acts of imagination. They’re ideas we play games with.
IDK, I feel like you could say the same sentences truthfully about math, and if you “went with the overall vibe” of them, you might be confused and mistakenly think math was “arbitrary” or “meaningless”, or doesn’t have a determinate tendency, etc. Like, okay, if I say “one element of moral progress is increasing universalizability”, and you say “that’s just the thing your status cohort assigns high status”, I’m like, well, sure, but that doesn’t mean it doesn’t also have other interesting properties, like being a tendency across many different peoples; like being correlated with the extent to which they’re reflecting, sharing information, and building understanding; like resulting in reductionist-materialist local outcomes that have more of material local things that people otherwise generally seem to like (e.g. not being punched, having food, etc.); etc. It could be that morality has tendencies, but not without hormesis and mutually assured destrubtion and similar things that might be removed by aligned AI.
Hold on, I guess this actually means that for a natural interpretation of “entropy” in “generic uncertainty about maybe being wrong, without other extra premises, should increase the entropy of one’s probability distribution over AGI,” that statement is actually false. If by “entropy” we mean “entropy according to the uniform measure”, it’s false. What we should really mean is entropy according to one’s maximum entropy distribution (as the background measure), in which case the statement is true.
I have calculated the number of computer operations used by evolution to evolve the human brain—searching through organisms with increasing brain size - by adding up all the computations that were done by any brains before modern humans appeared. It comes out to 10^43 computer operations. AGI isn’t coming any time soon!
And yet, because your reasoning contains the word “biological”, it is just as invalid and unhelpful as Moravec’s original prediction.
I agree that the conclusion about AGI not coming soon is invalid, so the following isn’t exactly responding to what you say. But: ISTM the evolution thing is somewhat qualitatively different from Moravec or Stack More Layers, in that it softly upper bounds the uncertainty about the algorithmic knowledge needed to create AGI. IDK how easy it would be to implement an evolution that spits out AGI, but that difficulty seems like it should be less conceptually uncertain than the difficulty of understanding enough about AGI to do something more clever with less compute. Like, we could extrapolate out 3 OOMs of compute/$ per decade to get an upper bound: very probably AGI before 2150-ish, if Moore’s law continues. Not very certain, or helpful if you already think AGI is very likely soon-ish, but it has nonzero content.
Now having read the rest of the essay… I guess “maximum entropy” is just straight up confusing if you don’t insert the ”...given assumptions XYZ”. Otherwise it sounds like there’s such a thing as “the maximum-entropy distribution”, which doesn’t exist: you have to cut up the possible worlds somehow, and different ways of cutting them up produces different uniform distributions. (Or in the continuous case, you have to choose a measure in order to do integration, and that measure contains just as much information as a probability distribution; the uniform measure says that all years are the same, but you could also say all orders of magnitude of time since the Big Bang are the same, or something else.) So how you cut up possible worlds changes the uniform distribution, i.e. the maximum entropy distribution. So the assumptions that go into how you cut up the worlds, are determining your maximum entropy distribution.
(I’m taking the tack that “you might be wrong” isn’t just already accounted for in your distributions, and you’re now considering a generic update on “you might be wrong”.)
so you’re more confident about your AGI beliefs, and OpenPhil is less confident. Therefore, to the extent that you might be wrong, the world is going to look more like OpenPhil’s forecasts of how the future will probably look
Informally, this is simply wrong: the specificity in OpenPhil’s forecasts is some other specificity added to some hypothetical max-entropy distribution, and it can be a totally different sort of specificity than yours (rather than simply a less confident version of yours).
Formally: It’s true that if you have a distribution P, and then update on “I might be wrong about the stuff that generated this distribution” to the distribution P’, then P’ should be higher entropy than P; so P’ be more similar in that it’s higher entropy to other distributions Q with higher entropy than P. That doesn’t mean P’ will be more similar than P in terms of what it says will happen, to some other higher entropy distribution Q. You could increase the entropy of P by spreading its mass over more outcomes that Q thinks are impossible; this would make P’ further from Q than P is from Q, on natural measures of distance, e.g. KL-divergence (quasi-metric) or earth-mover or whatever. (For the other direction of KL divergence, you could have P reallocate mass away from areas Q thinks are likely; this would be natural if P and Q semi-agreed on a likely outcome, so that P’ is more agnostic and has higher expected surprise according to Q. We can simultaneously have KL(P,Q) < KL(P’,Q) and KL(Q,P) < KL(Q,P’).)
(Also I think for basically any random variable X we can have |E_P(X) - E_Q(X)| < |E_P’(X) - E_Q(X)| for all degrees of wrongness giving P’ from P.)
If you put higher probabilities on AGI arriving in the years before 2050, then, on average, you’re concentrating more probability into each year that AGI might possibly arrive, than OpenPhil does.
This is true for years before 2050, but not necessarily for years after 2050, if your distribution e.g. has a thick tail and OpenPhil has a thin tail. It’s true for all years if both of your distributions are just constant probabilities in each year, and maybe for some other similar kinds of families.
Your probability distribution has lower entropy [than OpenPhil’s].
Not true in general, by the above. (It’s true that Eliezer’s distribution for “AGI before OpenPhil’s median, yea/nay?” has lower entropy than OpenPhil’s, but that would be true for any two distributions with different medians!)
So to the extent that you’re wrong, it should shift your probability distributions in the direction of maximum entropy.
This seems right. (Which might be away from OpenPhil’s distribution.) The update from P to P’ looks like mixing in some less-specific prior. It’s hard to say what it should be; it’s supposed to be maximum-entropy given some background information, but IDK what the right way is to put a maximum entropy distribution on the space of years (for one thing, it’s non-compact; for another, the indistinguishability of years that could give a uniform distribution or a Poisson distribution seems pretty dubious, and I’m not sure what to do if there’s not a clear symmetry to fall back on). So I’m not even sure that the median should go up!
Is Humbali right that generic uncertainty about maybe being wrong, without other extra premises, should increase the entropy of one’s probability distribution over AGI,
Yes.
(I mean, if what you think you’re maybe wrong about, is specifically some arguments that previously updated you to be less confident than some so-called “maximum”-entropy distribution, then you’d decrease your entropy when you put more weight on being wrong. This isn’t generic wrongness, since it’s failing to doubt the assumptions that went into the “maximum”-entropy distribution, which apparently you can coherently doubt, since previously some arguments left you to fall back on some other higher-entropy distribution based on weaker assumptions. But I guess it could look like you were supposed to fall back on the lower-entropy distribution, if that felt like the “background”.)
thereby moving out its median further away in time?
Not necessarily. It depends what the max-entropy distribution looks like, i.e. what assumptions you’re falling back on if you’re wrong.
+1
A relevant aspect of in-person interactions is that I think they involve a lot more “plasticity” of the people. In terms of how much B “is given write access” to C’s soul, it tends (with variance) to be something like (abstracting over content):
C reads B’s writing < C listens to B speaking < C listens to B and watches B acting < C is physically present with B < C is physically present with B and is speaking with B < C is physically present with B and is acting in concert with B
An example of what I mean by B having written to C’s soul is that C can “hear B’s voice” even when B isn’t there; e.g. C reflexively imagines what B would say about what C is doing. Or more abstractly, a proposition B said might bounce around in C’s head, being chewed on and propagated. B has somewhat literally made an impression on C. C might adopt mannerisms of B. C might do to D actions that imitate “deepening” (hence correlatedly subtly invasive or coercive or deceptive) actions done to C by B (because, oh, that’s how connection works, apparently).
(Obviously in general there’s huge mutual benefits to this soul-writing thing, which explains why people do it, which explains why it’s vulnerable to exploitation.)
I think I agree with ~everything in your two comments, and yet reading them I want to push back on something, not exactly sure what, but something like: look, there’s this thing (or many things with a family resemblance) that happens and it’s bad, and somehow it’s super hard to describe / see it as it’s happening.… and in particular I suspect the easiest, the first way out of it, the way out that’s most readily accessible to someone mired in an “oops my internal organs are hooked up to a vampiric force” situation, does not primarily / mainly involve much understanding or theorizing (at least given our collective current level of understanding about these things), and rather involves something with a little more of “wild” vibe, the vibe of running away, of suddenly screaming NO, of asserting meaningful propositions confidently from a perspective, etc. And I get some of this vibe from the OP; like part of the message is (what I’m interpreting to be) the stance someone takes when calling something “frame control” (or “gaslighting” or “emotional abuse” or “cult” or what-have-you).
Which, I still agree with the things you say, and the post does make lots of sort-of-specific, sort-of-vague claims, and gives good data with debatable interpretation, and so on. But there’s also this sort of necessarily pre-theoretic theoretic action happening, and I guess I want to somehow have that [hypothesis mixed with judgement mixed with action] be possible as well, including in the common space. (Like, the action is theoretic in that you’re reifying some pattern (e.g. “frame control”). It’s almost necessarily pre-theoretic, in the sense that you don’t even close to fully understand it and it’s probably only very roughly joint-carving, because the pattern itself involves making you confused about what’s happening and less able to clearly understand patterns. It’s an action, a judgement that something is really seriously wrong and you need to change it, a mental motion that rejects something previously accepted, that catapults you out of a satisficing basin; and you’re doing this action in a way that somewhat strongly depends or is helped by the non-joint-carving unrefined concept, like “this thing, IDK what it is really, but it’s really bad and I have to get out of it, and after escaping I’ll think about it more”.)
I see
youyour comments as partly rejecting, or at least incidentally pushing against, this sort of action: to “do it in a way that telegraphs the early-stage-ness” is, when speaking from a pre-theoretic standpoint, in tension with the vibe/action of sharply reclaiming one’s own perspective even when that perspective is noticeably incoherent (“something was deeply wrong, I don’t know what”). Like, it’s definitely a better artifact if you put in the right epistemic tags that point towards uncertainty, places to refine and investigate, etc.; but that’s harder to do and requires the author to be detailedly tracking a more complicated boundary around known and unknown, in a way that’s, like, not the first mental motion that (AFAIK) has to happen to get the minimum viable concept to self-coordinate on a narrative that says the thing is bad. Internally coordinating on a narrative that X-whatever-it-is is bad, seems important if you’re going to have to first push against X in big ways, before it’s very feasible to get a better understanding of X. (There’s bucket errors here, and it could be helpful to clarify that; but that’s maybe sort of the point: someone who’s been given a heavy dose of frame control is bucket-errored such that they doubt the goodness of holding their own perspective in part because it’s been tied up with other catastrophic things such as disagreeing with their social environment without having a coherent alternative or a coherent / legible grounds for disagreeing.)
I still haven’t had the issues come back I did prior to that reclaiming moment, and I’ve had no further detection of unhonored or ignored pain
Interesting, thanks for the data!
I’ll be curious to see your further writing.
burn frame control with fire
Well, setting a fire might require you to get too close; nuking it from orbit is maybe prudenter.
>Indeed this feels kind of epistemically hopeless to ever evaluate from the outside? I don’t really know what to do with this thought but it felt important to note.
Does seem good to note, and it would be nice to have more theory about this. We could upgrade our individual abilities to notice when we’re being frame controlled / etc.; we could upgrade our collective abilities to aggregate information about whether / how someone is systematically or intentionally harmful; we could close social niches that call up abusive behavior.
I think one piece of the puzzle might be something like:
(1) B can’t abuse C without C having the capacity to notice *at some point*. Maybe it’s in a year when C isn’t enmeshed in the situation; maybe it’s only after C has read other accounts of abuse, or other accounts about B specifically;
(2) If B is intentionally* abusing people, B will tend to abuse multiple people, or one person across long time periods. (I don’t know if this is true; it’s easy enough to imagine B abusing only one person, but it seems unlikely to be intentionional; why would B only use this strategy in one isolated situation?
(3) If B is effectively, skillfully abusing people, B will tend to abuse multiple people, or one person across long time periods. (Because how else would B be good at frame control / etc.? This might not be true because there’s skill transfer; e.g. B might do a lot of work that involves deeply understanding people, which is otherwise benign, but gives them the tools to deeply fuck with people in isolated circumstances.)
(4) If you’re on the lookout for frame control and such, it’s harder to have it happen to you. But being on the lookout is a lot of work.
To the extent that these are all true, ISTM it would be good to somehow be much more willing to publicly discuss stuff like this about specific people. Obviously there’s huge issues with scapegoating, and basically people lying. But, it seems that there’s value on the table, where giving public reports like “I felt harmed by my interaction with B”, being agnostic about intentionality or even causality, and without scapegoating on that basis, would allow future targets to effectively invest their limited capacity to detect stuff (and then make further reports, which opens us up to a sort of streetlight-bias, which we could hopefully correct for).
Another tack, is the idea of investigations—investigative journalism, or (cross-)examinations in a criminal trial. Simply having someone ask searching questions can reveal stuff that’s hard to pin down by default. I wonder how much abuse could be revealed with a series of three 3-hour interviews, or something.
(By “intentionally doing X” I don’t mean knowingly, conciously, deliberately, or endorsedly, but I do mean more than systematically. “B systematically does X” means simply that B tends to do X more than usual, more than some default, etc. If B intentionally does X, then B systematically does X. But say B is ugly, and people systematically treat ugly people in such a way that X is suitable behavior in that context; then I’d say B does X systematically, but only with very weak intentionality; the intentionality routes through other people, so it makes more sense to say that other people intentionally evoke X. By “B intentionally does X” I mean that X is an aim of B. The more super-ordinate the aim is, in a hierarchy of aims, the more intentional it is; the more B’s acheiving effect X is robust to situations, the more intentional it is.)
[Mostly unrelated but sparked by skimming this comment]
It occurs to me that another question around frame control, is: how can I / we facilitate social niches that don’t require frame control? In the leadership example: how can I be more willing and able to be led effectively by someone who is e.g. deeply and truly criticized in front of the group? For example, this might involve being more careful about not falling into misinformation cascades, and more intentional about hope.
I agree that’s a key question, though it’s plausible to me that the reframing is related to a lot of mental things, and so has lots of effects that I don’t understand. E.g. if the reframing involves in some sense giving up on justice (<- just a speculation) then it could be locally behaviorally right (justice may be too costly in this case) while also accidentally involving more broadly giving up on justice including where justice would be good.
AlphaFold 2 coming out of Deepmind and shocking the heck out of everyone in the field of protein folding with performance far better than they expected even after the previous shock of AlphaFold, by combining many pieces that I suppose you could find precedents for scattered around the AI field, but with those many secret sauces all combined in one place by the meta-secret-sauce of “Deepmind alone actually knows how to combine that stuff and build things that complicated without a prior example”?
Hm. I wonder if there’s a bet to be extracted from this. Like: Eliezer says that Alphafold 2 beats [algorithms previous to Alphafold 2, but with 10x compute], and Paul says the latter beats the former? Or replace Alphafold 2 with anything that Eliezer thinks contains some amount of secret sauce over previous things (whether or not its performance is “on trend”).
Here’s some notes I took about the first some minutes of Gaslight (1944) (SPOILER alert. It’s a very good movie, and somewhat relevant).
When he grabs the letter out of her hands he’s like “Oh uh I was just worried about all the unhappy memories it’s reminding you of”. It’s weird, it’s a double move: on the one hand, most obviously it’s a lie to cover up that he’s worried about something else, but also it reveals that he’s positioning himself as hyperconcerned about her. He doesn’t excuse it by some selfish motive like “I became super curious about the letter” or “Your talking is annoying me” or whatever. Further, his supposed concern is about her “unhappy memories”, positioning himself as an agent who takes it as a salient variable to track, what’s going on with her memories and emotions; and implicitly, that he’s an agent in the position to affect and manage her relationship with her memories and emotions.
And in the next breath, he explicitly tells her to forget all that unhappy stuff. He says “While you are afraid of anything, there cannot be any happiness for us”; “You must forget her”. This sounds sort of innocent, especially in the context of concern, but it’s ambiguous between a mere description/prediction of what will make them happy, vs. a threat of e.g. leaving or withholding happiness from her if she doesn’t follow his orders. It’s also just an obviously extreme (and implausible) statement when considered explicitly / from a third-person perspective, but in a way that could slip by as merely being high-intensity because the situation is high-intensity, rather than itself being a crazy statement.Her only response is to say “Well, not [to forget] her, but what happened to her”. Which, now that I think of it, makes the extremeness look intentional: by making an extreme statement and command, her corrective reasoning doesn’t correct far enough. She’ll remember her aunt, but has still tacitly agreed to forget what happened to her aunt; and this constitutes a step towards buying in to him being appropriate to give her orders about what to do with her mind.
He gives her his mother’s pin as a gift, but then immediately takes it back. “You are inclined to lose things Paula.” “I didn’t realize that.” “Oh just little things.” It’s a subtle dashing of hope; supposedly she’s special enough for him to give her his heirloom, and he loves her and wants to symbolize her specialness, but then he “realizes” that she’s unreliable; it’s her fault, so *in opposition* to the motions of his naive love for her he has to maturely handle her. He’s staying in the frame of concern that he’ll keep the pin for safekepping so she doesn’t lose it. He probes with a strong claim, and then when she notes her surprise, he “clarifies”, which is in effect a subtle retreat, to a weaker and more difficult to verify claim (“just little things”); it’s just on the edge where it’s reasonable to trust someone’s report, since maybe they noticed something you didn’t, and it’s hard to falsify. He’s using her feedback about when she notices that his claims are implausible, to calibrate where and how far he can go in each case.
“There, now you’ll remember where it is.” “Oh don’t be silly, of course I’ll remember.” “Oh I was teasing you my dear.” More probing and walking-back. He reasserts his proposition about her forgetfulness, by implication (“now you’ll remember [and you wouldn’t otherwise]”); and then he pretend to not have actually been doing so; and he also implicitly asserts that she can’t tell when he’s teasing her.
“You’ve been forgetting things. Don’t worry, you’ve been tired.” “Yes that’s it, I’ve been tired.” He gives her an explanation that she can be sure will be accepted by the shared narrative; thus defusing the tension of the falsification of her experience without her having to directly contradict his false report; and thus walking her further into committing to agreeing that she’s unreliable.He took the broach, causing her to believe she’d harmed him, then when she accepts that she’s been forgetful and is spooked and sad / guilty, he comforts her, pretending it’s just that she’s tired. It’s a smokescreen for him: he appears to be comforting, when he knows he can keep the ruse up, and that she’ll now believe the ruse and won’t suspect him. And really what’s happening is that he’s dispelling her localized, specific confusion—her confidence that something weird just happened because it contradicts her memory, her concentration of epistemic force in time, her attention on the details of a context in which a trick is in fact being played on her and she’s noticed and is trying to form a coherent hypothesis—he dispels that by giving her an explanation which tacitly still assumes that in fact what happened was that she was forgetful.
“Don’t worry.” More telling her what to feel. “It’s not valuable.” Knowing that she wasn’t worried that he was worried about the monetary value; then when she apologizes more because it’s his family hierloom, he doesn’t contradict her. That way the positive transcript—what he actually said out loud—reads like he’s being forgiving and saying that she doesn’t owe him, the negative transcript shows that he’s letting her insist that she’s at fault, harmed him. Reminds me of a dinner guest insisting on helping clear the table and the host super-insisting it’s fine; the guest is sometimes in a sense insincere. He sets it up so that it’s she, not he, who brings the true harm of losing an heirloom into the conversation; subtly this reinforces that she’s worse than he’s letting on, and it’s only his generosity that’s keeping them together, despite what she would know if only she checks to see that she’s bad.
“It hurts me when you’re ill and fanciful.” After confronting her, challenging her to assert—while he’s staring her down, after being cross with her and denying he’s cross with her—that she is actually perceiving that the maid despises her. “I hope you’re not starting to imagine things again.” “You’re not, are you Paula?” Either she’s starting to imagine things again, or the maid despises her; the former is a catastrophized version of the obvious other hypothesis, that she’s imagining / mistaken about just this particular thing. By catastrophizing, he removes reasonable options—either deny her husband’s narrative based on an uncertain perception, or else admit to being teetering on the edge of losing her mental continence.
“But my dear, I thought you were only being polite, why didn’t you tell me you really wanted to see her?” Right after he just yelled at her to get her to say no. He silences her, and pretends that he’s left communication channels perfectly well open if only she would just use them in the obvious way.
He lets her infer that she forgot he’s taking her out, lets her stew in that for like 20 seconds, talks about something else, then tells her it was indeed a surprise.
Et cetera.
(Damn, that dude really likes jewels.)
Thinking is like kicking a rock down a lane as you walk. If the object is oddly shaped, it may tumble oddly and go off in some oblique direction even if you impelled it forcefully straight. Without care, you’re liable to leave the object by the wayside and replace it with another, or with nothing. Tendencies of the object’s motion are produced both by the landscape—the slopes and the textures—and by the way you impel it, in big or little steps, with topspin or sidespin. The object may get stuck in a pothole or by the curb, and there’s no guarantee you’ll have the patience, or the small-foot-ness, needed to free it. People may look at you funny, as though you’re acting like a child, and you’re certain to leave the project behind without a bit of obliviousness and stand-offishness in you. It’s not clear whether you’re accomplishing anything with each step, let alone whether there is or what might be the ultimate payoff.