TekhneMakre

Karma: 1,416

TekhneMakre 15 May 2022 3:32 UTC
2 points
in reply to: johnswentworth’s comment on: The Big Picture Of Alignment (Talk Part 1)
If natural abstractions are a thing, in what sense is “make this AGI have particular effect X” trying to be about human values, if X is expressed using natural abstractions?

TekhneMakre 14 May 2022 18:09 UTC
4 points
in reply to: johnswentworth’s comment on: The Big Picture Of Alignment (Talk Part 1)
>Like, ability-to-narrow-down-a-search-space-or-behavior-space-by-a-factor-of-two is what a bit of information is.
Information is an upper bound, not a lower bound. The capacity of a channel gives you an upper bound on how many distinct messages you can send, not a lower bound on your performance on some task using messages sent over the channel. If you have a very high info-capacity channel with someone who speaks a different language from you, you don’t have an informational problem, you have some other problem (a translation problem).
>If we can’t use the information to narrow down a search space closer to the thing-the-information-is-supposedly-about, then we don’t actually have any information about that thing.
This seems to render the word “information” equivalent to “what we know how to do”, which is not the technical meaning of information. Do you mean to do that? If so, why? It seems like a misframing of the problem, because what’s hard about the problem is that you don’t know how to do something, and don’t know how to gather data about how to do that thing, because you don’t have a clear space of possibilities with a shattering set of clear observable implications of those possibilities. When you don’t know how to do something and don’t have a clear space of possibilities, the sort of pieces of progress you want to make aren’t fungible with each other the way information is fungible with other information.
[ETA: Like, if the space in question is the space of which “human values” is a member, then I’m saying, our problem isn’t locating human values in that space, our problem is that none of the points in the space are things we can actually implement, because we don’t know how to give any particular values to an AGI.]

TekhneMakre 14 May 2022 9:29 UTC
4 points
in reply to: johnswentworth’s comment on: The Big Picture Of Alignment (Talk Part 1)
What’s the search space? Policies, or algorithms, or behaviors, or something. What’s the information? Well, basically pointing a camera at anything in the world today gives you information about human values, or reading anything off the internet. What do we do with this information to get policies we like? The bits of information isn’t the problem, the problem is that we don’t know how to narrow down policy space or algorithm space or behavior space so that it has some particular final results. Getting bits of information about human values, and being able to aim an AGI at anything, are different problems.

TekhneMakre 14 May 2022 4:54 UTC
2 points
in reply to: johnswentworth’s comment on: The Big Picture Of Alignment (Talk Part 1)
So “get enough bits-of-information about human values” makes sense if you have something you can do with the bits, i.e. narrow down something. If we don’t know how to specify any final effect of an AGI at all, then we have an additional problem, which is that we don’t know how to do anything with the bits of information about which final effects we want.

TekhneMakre 14 May 2022 4:31 UTC
4 points
in reply to: johnswentworth’s comment on: The Big Picture Of Alignment (Talk Part 1)
Huh. I thought Yudkowsky’s view was that it’s hard to get an AGI to do X for all values of X, where X is the final effect of the AGI on the world (like, what the universe looks like when the AI is done doing its thing). If X is instead an instrumental sort of thing, like getting a lot of energy and matter, then it’s not hard to get an AGI to do that.

TekhneMakre 13 May 2022 9:01 UTC
2 points
on: The Big Picture Of Alignment (Talk Part 1)
Bump re/ my question about trying to make an AI do any specifiable thing at all vs. specifying some good thing to do; still curious what you think.

TekhneMakre 11 May 2022 2:40 UTC
2 points
on: Fighting in various places for a really long time
For me it started off promising because it seemed like it was taking the multiverse “seriously” (in some sense; not in the sense of making logical or physical sense, not in the sense of being non-humorous, but in the sense of the multiverse being Real for the characters). I thought it was poetically pregnant how the older woman was powerful because of all her unfollowed dreams.
The nihilism could hypothetically have been interesting, and could have worked with the trauma or whatever, but IMO it didn’t end up coherent / archetypally real / resonant. (I also thought the movie was very flawed in other ways that significantly detracted from it for me, e.g. most of the humor fell flat to me and the structure became very repetitive, like they made a spreadsheet of 4 alternate universes, 4 plot points, and then filled out every combination of alt + situation.)

TekhneMakre 8 May 2022 2:37 UTC
2 points
in reply to: Eli Tyre’s comment on: Optimality is the tiger, and agents are its teeth
This is an unnatural use of “most”. Extremely optimized outputs will tend to be dangerous, on their own, even if they are actually just optimized “for something”. It seems more natural to say that for most features such that you know how to ask for something to be very optimized on that feature, something extremely optimized for that feature will be dangerous.

TekhneMakre 7 May 2022 3:45 UTC
5 points
in reply to: Quintin Pope’s comment on: Is there a convenient way to make “sealed” predictions?
(Note that this has exactly the same potential security problems as something involving a key; e.g. if you keep a copy on a machine connected to the internet, someone might hack you and steal the secret message.)

TekhneMakre 2 May 2022 23:23 UTC
2 points
in reply to: ViktoriaMalyasova’s comment on: Narrative Syncing
> secret rule
It shouldn’t be secret.

The third answer gets it wrong if Alec takes it as an order as opposed to potentially useful information.
>Just figure out what information is useful for newcomers, and share it.
Yes, but this only makes sense if your statements are taken as information. If they aren’t, then the useful information is the fact that your statements aren’t being taken as information.

TekhneMakre 2 May 2022 9:23 UTC
3 points
in reply to: Daphne_W’s comment on: Narrative Syncing
That seems maybe right if Alec isn’t *interested* in helping in non-”grunt” ways. (TBC “grunt” stuff can be super important; it’s just that we seem much more bottlenecked on 1. non-grunt stuff, and 2. grunt stuff for stuff that’s too weird for people like this to decide to work on.) I’m also saying that Alec might end up being able and willing to help in non-grunt ways, but not by taking orders, and rather by going off and learning how to do non-grunt stuff in a context with more clear feedback.
It could be harmful to Alec to give him orders to work on “grunt” stuff, for example by playing in to his delusion that doing some task is crucially important for the world not ending, which is an inappropriate amount of pressure and stress and more importantly probably is false. It could potentially be harmful of Alec if he’s providing labor for whoever managed to gain control of the narrative via fraud, because then fraudsters get lots of labor and are empower to do more fraud. It could be harmful of Alec if he feels he has to add weight to the narrative that what he’s doing matters, thereby amplifying information cascades.

TekhneMakre 1 May 2022 22:40 UTC
2 points
in reply to: ambigram’s comment on: Narrative Syncing
Anyone can say “I’m an expert”. There has to be some other way that you’re distinguishing who’s a newbie who’s just clueless, from an expert who’s ignorance is evidence of a general difficulty. From my perspective, Berry is helping cooperation with Alec by just making straightforward statements from Berry’s own perspective; then Alec can compare what Berry says with what other people say and with what seems to Alec to match reality and be logically coherent, and then Alec can distinguish who does and doesn’t have informed, logically coherent opinions refined through reason. E.g., at a gloss, Yudkowsky says “no one has a plan that could possibly work”, and Christiano says “no, I have a plan which, with some optimistic but not crazy assumptions, looks like it’ll probably work”, and Yudkowsky says something that people don’t understand, and Christiano responds in a way that may or may not address Yudkowsky’s critique, and who knows who’s an expert? Just going by who says “I have a plan” is helpful if someone has a plan that will work (although it also opens up social niches for con artists).

TekhneMakre 1 May 2022 21:58 UTC
2 points
in reply to: Daphne_W’s comment on: Narrative Syncing
Good point. My guess is that if Alec is sufficiently like this, the right thing to do is to tell Alec not to work on AI risk for now. Instead, Alec, do other fun interesting things that matter to you; especially, try looking for things that you’re interested in / matter to you apart from social direction / feedback (and which aren’t as difficult as AI safety); and stay friends with me, if you like.

TekhneMakre 1 May 2022 5:12 UTC
3 points
in reply to: ambigram’s comment on: Narrative Syncing
>my true question (I still don’t know what to do next!)
(The following might be a strawman; sorry if so; but it is a genuine question about a phenomenon I’ve seen.) This part sounds like you’re saying, you’re asking someone what to do next. If so, what if it’s a mistake for you to be mentally executing the procedure 1. hear what someone says to do next, 2. do that thing? How would the person you’re asking communicate to you that it’s a mistake for you to execute that procedure, without you taking their attempted statements of fact as cryptic instructions for you to follow? (Assuming that’s a mistake you think you could plausibly be making, and assuming you’d want the mistake communicated to you if you were making it.)
>in this context it sounds like you’re refusing to help me (you’re not playing your role of advisor/expert?).
Someone’s role is a combination of their behavior and the behavior of others’ behavior and expectations. Suppose that Berry wants to invite Alec to a workshop with an offer to help, but *not* with any implied intent to play a role of advisor/expert. How could Berry communicate that to Alec?
>What cooks do is that they try adding a bit first (e.g. half a teaspoon) then taste and make adjustments.
In the case of alignment stuff, no one knows how to cook, so the version of this that’s available is “try to figure the whole thing out for yourself with no assumptions”, which was part of Answer 1.

TekhneMakre 1 May 2022 3:04 UTC
4 points
on: Narrative Syncing
>Note that in this example, the “me” character does do narrative syncing – but in a way that does not disguise itself as information sharing about AI.
Well, it’s better, but in I think you’re still playing into [Alec taking things you say as orders], which I claim is a thing, so that in practice Alec will predictably systematically be less helpful and more harmful than if he weren’t [taking things you say as orders].
>How can a person shift things toward less confusing patterns?
[Tossing out ideas, caveat emptor]
You could draw attention in the moment to that dynamic, so that Alec can get introspection on the phenomenon, or metaphorically so that Alec can start to get a “brain-computer interface readout” on “right now I’m looking to be told what to do”.
You could abstractly clarify the situation (which this post helps with, thank you!).
You could do improv-y stuff, playing low status, playing like you’re looking for orders from Alec. (Warning, might be a kind of deception.)The supposed upside is to “shake things loose” a little so that Alec gets exposed to what it would be like if he was High and you were Low, so that when he’s back to his usual Low play, he can know it’s happening and know what’s different about it.
You could refuse to answer Alec until it seems like he’s acting like his own boss.
You could technically explain why you don’t want to give him orders.
You could address why Alec locally wants to be told what to do.

>Is it something we should simply try to minimize if we want good epistemics?
My wild guess is that first-order minimizing narrative-syncing isn’t that important, but being open to second-order corrections is very important, because that’s what determines your trajectory towards correct beliefs vs. delusion. There’s always a ton of narrative-syncing taken for granted, and primordially speech is corrections to that; just-trying-to-be-epistemic speech is a rarefied sort of thing. I don’t know how to balance being open to corrections against “losing momentum”, but the obvious guess is to not sacrifice any ability to correct at all, because subtle failures-to-correct can propagate as a narrative-sync to just not think about something. In other words, yay to simply minimizing pushes against correcting errors, boo to minimizing narrative-syncing.

>What’re some places where this happens a lot?
This happens a lot with a power / status differential. This isn’t a vacuous claim: one consequence is that I’m claiming that for each person Alec, there’s a roughly 1-dimensional value called “power” (well, maybe it depends on social context...) that Alec assigns to each other person Berry, and this “power” variable predicts all of: the extent to which Alec imputes narrative-syncingness onto Berry’s utterances; the extent to which Alec expects others to take Blob’s utterances as narrative-syncing; the extent to which Alec expects Berry to take others’s statements as narrative-syncing; the extent to which Alec, if so inclined, to punish deviations from those expectations.
>How can a person notice it?
If Berry “directs annoyance at” Alec in a way where the annoyance isn’t hidden but the reasons for the annoyance are hidden, that might be because Berry expected Alec to coordinate in a narrative-sync but Alec didn’t.
If Alec finds himself doing stuff on automatic, not really interested / curious / etc., that might be because he took orders from someone.
> “Why did I even sign up for this program?”
To say something maybe obvious, this seems like a great chance to pry your fingertips under the shield of narratives; why are you here Alec? You were expecting me to do something or know something? Here’s a description of some of the tacks I’ve taken in trying to solve this thing, and why they can’t work; what should we do?

TekhneMakre 26 Apr 2022 23:17 UTC
2 points
in reply to: Jeff Rose’s comment on: Rationalist Should Win. Not Dying with Dignity and Funding WBE.
The people you list are all in contexts where they are forced to juggle many forces having different goals, capacity for violence, sizes, etc. Leaders like these gain and keep power by {cajoling, rewarding, punishing, threatening} the military, capitalists, businessmen, revolutionaries, farmers, peasants, bureaucrats, and foreign nations; and by making themselves Basilisks of violence (I think that’s why Hitler and Mussolini loved each other). It’s not obvious to me that these forces *made* e.g. Hitler what he was, but they seem pretty different in nature, and plausibly also in effect, from the incentives of a genius in a vat in a basement with a few genius buddies with a mission to save the world.

TekhneMakre 25 Apr 2022 6:42 UTC
8 points
on: Report likelihood ratios
Cf. https://www.overcomingbias.com/2009/02/share-likelihood-ratios-not-posterior-beliefs.html

TekhneMakre 21 Apr 2022 3:41 UTC
3 points
in reply to: jessicata’s comment on: MIRI announces new “Death With Dignity” strategy
Glad this is shared.
Jess: But they don’t share them because “danger” so no one can check their work, and it looks like a lot of nothing from the outside.
X: It’s a shocking failure of rationality.
Jess: Yes.
There’s an awkward issue here, which is: how can there be people who are financially supported to do research on stuff that’s heavily entangled with ideas that are dangerous to spread? It’s true that there are dangerous incentive problems here, where basically people can unaccountably lie about their private insight into dangerous issues; on the other hand, it seems bad for ideas to be shared that are more or less plausible precursors to a world-ending artifact. My understanding about Eliezer and MIRI is basically, Eliezer wrote a bunch of public stuff that demonstrated that he has insight into the alignment problem, and professed his intent to solve alignment, and then he more or less got tenure from EA. Is that not what happened? Is that not what should have happened? That seems like the next best thing to directly sharing dangerous stuff.
I could imagine a lot of points of disagreement, like
1. that there’s such a thing as ideas that are plausible precursors to world-ending artifacts;
2. that some people should be funded to work on dangerous ideas that can’t be directly shared / evidenced;
3. that Eliezer’s public writing is enough to deserve “tenure”;
4. that the danger of sharing ideas that catalyze world-ending outweighs the benefits of understanding the alignment problem better and generally coordinating by sharing more.
The issue of people deciding to keep secrets is a separate issue from how *other people* should treat these “sorcerers”. My guess is that it’d be much better if sorcerers could be granted tenure without people trusting their opinions or taking instructions from them, when those opinions and instructions are based on work that isn’t shared. (This doesn’t easily mesh with intuitions about status: if someone should be given sorcerer tenure, isn’t that the same thing as them being generally trusted? But no, it’s not, it should be perfectly reasonable to believe someone is a good bet to do well within their cabal, but not a good bet to do well in a system that takes commands and deductions from hidden beliefs without sharing the hidden beliefs.)

TekhneMakre 5 Apr 2022 8:10 UTC
7 points
on: What an actually pessimistic containment strategy looks like
And the best part of this is that instead of telling people who might already be working in some benign branch of ML that there’s this huge problem with AGI, who can potentially defect and go into that branch because it sounds cool, you’re already talking to people who, from your perspective, are doing the worst thing in the world. There’s no failure mode where some psychopaths are going to go be intrigued by the “power” of turning the world into paperclips. They’re already working at DeepMind or OpenAI. Personally, I think that failure mode is overblown, but this is one way you get around it.
If this is saying that there’s no plausible downside, that statement seems incorrect. It’s not a very important bit whether or not someone has a narrative of “working on AGI”. It takes 2 minutes to put that on a resume, and it would even be defensible if you’d ever done anything with a neural net. More important is the organizing principles of their technical explorations. There’s a whole space of possible organizing principles. If you’re publicizing your AI capabilities insights, the organizing principle of “tweak and mix algorithms to get gains on benchmarks” is less burning the AGI fuse than the organizing principle of “try new algorithms” is less burning the AGI fuse than the organizing principle of “try to make AGI”. To argue that AGI kills you by default, you argue that there are algorithmic inventions to be found that generalize across domains to enable large capability jumps, without having a controllable / understood relationship to goals. Which a fortiori says something about the power of generalization. Which might change how people organize their research. If “X is possible” can be the main insight of a research breakthrough, then “AGI is dangerous” contains “AGI is powerful” contains “power is possible”, and could have similar effects.
On another note:
There’s other downsides here. I don’t think that means this class of strategy should be taboo, and I think this class of strategy absolutely should be worked on. But pursuing a strategy without noticing the downsides, and noticing the ways the strategy is doomed to not actually help at all, is pretty crucial.
Downside: depending on implementation, you turn yourself into an ideological repeater. (This means you probably end up talking to people selected for, at least while they’re talking to you, being themselves ideological repeaters, which makes your strategy useless.) So you cause top AGI researchers to be more in an environment filled with ideological repeaters. So you cause it to be correct for top AGI researchers to model people around them as not having coherent perspectives, but instead to be engaging in ideological power struggles. If top AGI researchers are treating people that way, that bodes poorly for mind-changing communication.

TekhneMakre 4 Apr 2022 2:55 UTC
2 points
in reply to: TurnTrout’s comment on: Do a cost-benefit analysis of your technology usage
“participation in networks”? IDK. Is “social media” too narrow? I intuitively include LW, blogs, forums, email, as social media. Youtube is a mix; I consider it social media when I’m using it as pica for talking to a person, and I endorse most of my other Youtube use (e.g. videos of natural phenomena, or documentaries, or clever inventions). But yeah I could imagine being seriously attention-sniped in some other unendorsed way.
“attention service usage”? “consumption service usage”? That’s not really clear, though it does have a precise meaning that I think is what you’re trying to point at: services (i.e. computer programs running somewhere else that do some computing task for you) which are attention / consumption based (i.e. the thing you’re getting from the service is something you’re going to just directly experience, as opposed to running it through some more computation). Services which are attention-based seem like the central type of technology that induces attention-sniping incentives. (Though it’s not sufficient, e.g. Wikipedia.)