As I’ve tried to explain over and over, including once to Conor, if you want to improve thinking (rather than, say, “knowledge management” (text snippet / link management?)), you have to watch thinking think, think about how thinking thinks, and ask thinking what it would need in order to think better. No one who sets out to build so-called “tools for thinking” ever does this. They instead think of cool-sounding things to have, and then get excited imagining how those things might free your thoughts from the nested directory structure or whatever, and come up with unassailable arguments about that.
TsviBT(Tsvi Benson-Tilsen)
I like the essay and I think [something like what you call deep honesty] is underrated right now. But I’m still confused what you mean, and about the thing itself.
I’ll say a few more things but the headline is that I’m confused and would like more clarity about what a deep honesty-er is.
There’s always multiple audiences. A simple example is that anyone could repeat anything you say to someone else. A harder example is that individual humans are actually dividual.
“you can always be passively deeply honest with all of them” This is incorrect. They don’t speak the same language, and there are always many homonyms.
It’s not clear to me that it makes sense to at all think of bureaucracies as being the sort of thing that you can be honest or dishonest with—too schizophrenic / antiphrenic. Honesty, as you’ve described, is about putting more true + less false salient propositions in a mind as beliefs. There has to be a mind there to have propositions.
The essay is vague about who is benefiting. Which matters because the definition of deep honesty involves salience, which means it’s dependent on goal-pursuits or something else which gives salience to propositions. As Vassar said: As Kant said: What information architectures can I and should I integrate into?
Basically, I think we’re a lot less clear on [** what sort of being we would have to be for it to make sense to describe us as being (deeply) honest or not, or as being treated with (deep) honesty or not] than we should be.
Compared to an exhortation to deep honesty, I’m as much or more inclined to make an exhortation to figure out [** what sort of being...] and [what sort of being we would have to be for it to make sense for others to treat us with (deep) honesty]. Others fail me in both respects, but more so by not being suitable partners for deep honesty than by not being deeply honest.
Probabilities on summary events like this are mostly pretty pointless. You’re throwing together a bunch of different questions, about which you have very different knowledge states (including how much and how often you should update about them).
This practice doesn’t mean excusing bad behavior. You can still hold others accountable while taking responsibility for your own reactions.
Well, what if there’s a good piece of code (if you’ll allow the crudity) in your head, and someone else’s bad behavior is geared at hacking/exploiting that piece of code? The harm done is partly due to that piece of code and its role in part of your reaction to their bad behavior. But the implication is that they should stop with their bad behavior, not that you should get rid of the good code. I believe you’ll respond “Ah, but you see, there’s more than two options. You can change yourself in ways other than just deleting the code. You could recognize how the code is actually partly good and partly bad, and refactor it; and you could add other code to respond skillfully to their bad behavior; and you can add other code to help them correct their behavior.”. Which I totally agree with, but at this point, what’s being communicated by “taking self-blame” other than at best “reprogram yourself in Good/skillful ways” or more realistically “acquiesce to abuse”?
IDK if this is a crux for me thinking this is very relevant to stuff on my perspective, but:
The training procedure you propose doesn’t seem to actually incentivize indifference. First, a toy model where I agree it does incentivize that:
On the first time step, the agent gets a choice: choose a number 1--N. If the agent says k, then the agent has nothing at all to do for the first k steps, after which some game G starts. (Each play of G is i.i.d., not related to k.)
So this agent is indeed incentivized to pick k uniformly at random from 1--N. Now consider:
The agent is in a rich world. There are many complex multi-step plans to incentivize agent to learn problem-solving. Each episode, at time N, the agent gets to choose: end now, or play 10 more steps.
Does this incentivize random choice at time N? No. It incentivizes the agent to choose randomly End or Continue at the very beginning of the episode, and then carefully plan and execute behavior that acheives the most reward assuming a run of length N or N+10 respectively.
Wait, but isn’t this success? Didn’t we make the agent have no trajectory length preference?
No. Suppose:
Same as before, but now there’s a little guy standing by the End/Continue button. Sometimes he likes to press button randomly.
Do we kill the guy? Yes we certainly do, he will mess up our careful plans.
I think it’s a norm if you’re bidding for specific attention, yeah. Like, you should either do more work to figure out a smaller set of people who you want to bid for attention from, or else at least say that you haven’t done that work.
The examples in this comment are about “oops I had an idea that sounds good but is accidentally bad”. That’s a reasonable thing to worry about but doesn’t seem like the thing you were actually asking about. You wrote:
I don’t expect to be particularly good at coordinating with my perfect clones for example. I’m sure if you put me in a room with my perfect clone and a source of massive power (such as a controllable ASI), we’d beat each other half to death fighting for it.
This seems much more central, and indicates a major problem.
Is this referring to my insights in particular or something similar somebody else said?
It’s meant to gesture at a category of thinking, a given instance of which may or may not be worthwhile or interesting, but which leads people to be very overly worried about the consequences of spreading the ideas involved, compared to how bad the consequences actually are. For example, sometimes [people who take hypothetical possibilities very seriously] newly think of something, such as the potential of BCIs or the potential of thinking in such-and-such unconventional way or whatever. Then they implicitly reason like this: There’s a bunch of potential here; previously I hadn’t thought of this idea; previously I hadn’t pursued efforts related to this idea; now I’ve thought of this idea; the fact that I just now thought of the idea and hadn’t previously explains away the fact that I haven’t previously pursued related efforts; so probably my straightforward inside view of why there’s potential here is correct or at least a good rough draft guess; which means there are huge implications here; and the reason others aren’t pursuing related efforts is probably that they didn’t think of the idea; and since the idea is powerful, I shouldn’t share it.
Usually some but not all of these inferences are correct. Often the neglectedness is mainly because others don’t believe in hypothetical possibilities, not because no one has thought of it. Rarely does the final inference go through.
I’ve already had conversations with multiple billionaires.
I would think the problem here would be failing at transfering the relevant info, not transfering too much info!
But if you manage to get their attention you could get them to copy your preferred choices instead.
The only morally acceptable thing to copy in this way is an orientation against making decision this way.
For the most part, if you have a reason to share some information, you should share it. For the most part, trying to make a bunch of information boundaries will cripple your ability to do anything useful, and doesn’t avert much bad stuff. Your amazing strategic insights about how we’re all swimming in a sea of hyperstitious memetic warfare and therefore we can control the future by blah blah are usually false, and not actually that big if true because in general things are more in equilibrium than they seem and more driven by forces you’re not controlling than they seem. The more open I am about things I thought I should be cagey about, the more I find no one cares. Unless you’ve got a lot of attention for some reason, roughly no one cares about what you think enough to do much of anything in response to what you think.
There are obvious exceptions, like not sharing other people’s personal info in public or not sharing your garage nuke technology.
Distinguish [trust to not harm you, e.g. by misusing info you’ve shared] from [trust to meet your efforts toward a shared goal]. The latter is generally more important than the former, because lifeforce is a pretty limited resource, so you have to know where to invest yours.
I think it’s pretty defecty to email a lot of people without saying in the email that you’ve done so.
Bad restaurants are more likely to have open tables than good restaurants.
That seems dependent on it being difficult to scale the specific skill that went into putting together the experience at the good restaurant. Things that are more scalable, like small consumer products, can be selected to be especially good trades (the bad ones don’t get popular and inexpensive).
Bruh. Banana Laffy Taffy is the best. Happy to trade away non-banana to receive banana, 1:1.
The point of the essay is to describe the context that would make one want a hyperphone, so that
-
one can be motivated by the possibility of a hyperphone, and
-
one could get a hold of the criteria that would direct developing a good hyperphone.
The phrase “the ability to branch in conversations” doesn’t do either of those.
-
Quoting another comment I made:
Make a hyperphone. A majority of my alignment research conversations would be enhanced by having a hyperphone, to a degree somewhere between a lot and extremely; and this is heavily weighted on the most hopeworthy conversations. (Also sometimes when I explain what a hyperphone is well enough for the other person to get it, and then we have a complex conversation, they agree that it would be good. But very small N, like 3 to 5.)
Yes.
It’s a makeshift stop-gradient. I less feel like I’m writing to LessWrong if I’m not publishing it immediately, and although LW is sadly the best place on the internet that I’m aware of, it’s very much not in aggregate a gradient I want. Sometimes I write posts intended for LW and publish them immediately.
Make a hyperphone. A majority of my alignment research conversations would be enhanced by having a hyperphone, to a degree somewhere between a lot and extremely; and this is heavily weighted on the most hopeworthy conversations. (Also sometimes when I explain what a hyperphone is well enough for the other person to get it, and then we have a complex conversation, they agree that it would be good. But very small N, like 3 to 5.)
I’m not sure I understand your question at all, sorry. I’ll say my interpretation and then answer that. You might be asking:
Is the point of the essay summed up by saying: ” “Thing=Nexus” is not mechanistic/physicalist, but it’s still useful; in general, explanations can be non-mechanistic etc., but still be useful, perhaps by giving a functional definition of something.”?
My answer is no, that doesn’t sum up the essay. The essay makes these claims:
There many different directions in conceptspace that could be considered “more foundational”, each with their own usefulness and partial coherence.
None of these directions gives a total ordering that satisfies all the main needs of a “foundational direction”.
Some propositions/concepts not only fail to go in [your favorite foundational direction], but are furthermore circular; they call on themselves.
At least for all the “foundational directions” I listed, circular ideas can’t be going in that direction, because they are circular.
Nevertheless, a circular idea can be pretty useful.
I did fail to list “functional” in my list of “foundational directions”, so thanks for bringing it up. What I say about foundational directions would also apply to “functional”.
Hm, ok, thanks. I don’t I fully understand+believe your claims. For one thing, I would guess that many people do think and act, under the title “Buddhism”, as if they believe that desire is the cause of suffering.
If I instead said “Clinging/Striving is the cause of [painful wheel-spinning in pursuit of something missing]”, is that any closer? (This doesn’t really fit what I’m seeing in the Wiki pages.) I would also say that decompiling clinging/striving in order to avoid [painful wheel-spinning in pursuit of something missing] is tantamount to nihilism. (But maybe to learn what you’re offering I’d have to do more than just glance at the Wiki pages.)
Huh? A hyperphone is a two-player tool. Loom is a one-player tool.