Oh right sorry. Yeah, exactly.
TsviBT(Tsvi Benson-Tilsen)
Karma: 1,688
Shell games
I don’t recall seeing that theory in the first quarter of the book, but I’ll look for it later. I somewhat agree with your description of the difference between the theories (at least, as I imagine a predictive processing flavored version). Except, the theories are more similar than you say, in that FIAT would also allow very partial coherentifying, so that it doesn’t have to be “follow these goals, but allow these overrides”, but can rather be, “make these corrections towards coherence; fill in the free parameters with FIAT goals; leave all the other incoherent behavior the way it is”. A difference between the theories (though I don’t feel I can pass the PP ITT) is that FIAT allows, you know, agency, as in, non-myopic goal pursuit based on coherent-world-model-building, whereas PP maybe strongly hints against that?
It seems like the thing to do is to look for cases where people pursue their own goals, rather than the goals they would predict they have based on past actions.
I’m confused by this; are these supposed to be mutually exclusive? What’s “their own goals”? [After thinking more: Oh like you’re saying, here’s what it would look like to have a goal that can’t be explained as a FIAT goal? I’ll assume that in the rest of this comment.]
It needs to be complex enough to not plausibly be a reflex/instinct.
Agreed.
A sort of plausible example is courtship. It’s complex, it can’t easily be inferred from previous things you did (not the first time you do it, that is), and it agentically orients toward a goal.
I’m not sure I buy that it can’t be inferred, even the first time. Maybe you have fairly built-in instincts that aren’t about the whole courtship thing, but cause you to feel good when you’re around someone. So you seek being around them, and pay attention to them. You try to get them interested in being around you. This builds up the picture of a goal of being together for a long time. (This is a pretty poor explanation as stated; if this explanation works, why wouldn’t you just randomly fall in love with anyone you do a favor for? But this is why it’s at least plausible to me that the behavior could come from a FIAT-like thing. And maybe that’s actually the case with homosexual intercourse in the 1800s.)
The problem is, I think it’s well-explained as imitation—“I’m a person; the people around me do this and seem really into it; so I infer that I’m really into it too”.
Maybe courtship is especially much like this, but in general things sort-of-well-explainable as imitation seem like admissible falsifications of FIAT, e.g. if there are also pressures against the behavior.
It’s giving an alternative explanation of the observation.
Are there cognitive realms?
Thanks. Your comments make sense to me I think. But, these essays are more like research notes than they are trying to be good exposition, so I’m not necessarily trying to consistenly make them accessible. I’ll add a note to that effect in future.
Do humans derive values from fictitious imputed coherence?
I’m curious about what effects that had on you and your classmates. E.g., did it seem to cause an experience with math, that then seemed to cause further patterns of behavior later on?
Good to hear.
I do think there is some good urgent real thinking going on, that some people are a good fit for it, and can make a reasonable choice to do less serious play.
Definitely. A second-order hope I have is to make more space available for people to be more head-down intense about urgent thinking. The idea being:
Alice wants to be heads-down intense about urgent thinking. She does so. Then Bob sees Alice, and feels pressured to also be intense in that way. When Bob tries to be intense in that way, he throws away his mind, and that’s not good for him. He doesn’t fully understand what’s going wrong, but he knows that what’s going wrong has something to do with him being pressured to be intense. He correctly identifies that the pressure is partly caused by Alice (whether or not it’s Alice who really ought to change her behavior). Not understanding the situation in detail, Bob only has blunt actions available; and not having an explicit justification for pushing back against some pressure he feels, he doesn’t push back explicitly, out in the open, but instead puts some of his force toward implicitly pressuring Alice to not be so intense.
If Bob were more able to defend important things from implicit pressure that he feels, he’s less pushed to pressure Alice to not be intense. And so Alice is more freed to be intense, as is suitable for her.
(This is fairly theoretical, but would explain some of my experiences.)
If you happen to find something that stimulates some of your friends to try that sort of thing, I’d be curious to hear (or to read a blog post about). From my perspective it’s a big cultural lack.
In my observations in the wild, flow out of drainpipes created an A, and flow over the edge of a wall created a V. That seems intuitively to be something about: the drainpipe has a bunch of water all close together, and then it opens up with more space along the surface of the wall. The water flowing over the edge of the wall is spread out, and then wants to stick together to other water, so it gets pulled closer, forming a V, like how a fountain with a watersheet output at the top gives a V-shaped watersheet. I don’t feel completely satisfied by this though, because I don’t understand exactly why it spreads out when coming out of a drainpipe, rather than continuing to stick together.
Counting-down vs. counting-up coherence
I don’t get it yet. Thanks though! Might come back. (I’m probably not devoting enough attention to get it.)
We might have different things in mind with “intellectual inquiry”; depth is important. The first one seems like a seed of something that could be interesting. Phenomenology is the best data we have about real minds.
But mainly I made that comment because I don’t see insights from physics being “obviously applicable to working on alignment”. (This is maybe a controversial take and I haven’t thought about it that much and it could be stupid. I might also do accounting different, labeling more things as being “really math, not physics”.)
Does novel understanding imply novel agency / values?
Thanks.
the Testimony of a Cyborg appendix of the Cyborgism post
Nice!
<3
See my comment on the parent.
undercuts the message to “follow your playful impulses, even if they’re silly”
That’s a fine message, but it’s not the message of the post. The concept described in the post is playful thinking, not fun. It does use the word “fun” in a few places where the more specific phrase would arguably have been better, so the miscommunication is probably my fault.
are far less obviously applicable to working on alignment than the insights generated from studying physics
I literally don’t believe this, but even if it were true, the post doesn’t argue to do math because it’s useful for alignment. The reason that alignment is even mentioned, is that the phenomenon of throwing away your mind is so perverse that it should be obviously bad even for someone who only cares about alignment, or at least thinks they want to sacrifice their wellbeing for the sake of alignment; and because people around X-risk have a tendency to throw their mind away anyway. If someone tells me they decided not to take a 2 year surfing vacation because alignment is too urgent, then I might worry a bit about how that sits in them, but the decision makes sense to me and seems sane. If someone tells me they stopped thinking about stuff they’re really curious about because they don’t see how it helps alignment, I’m like “No, what the hell are you doing, who told you that was a good idea??”.
Thanks.
You make an important point. Fun in general is a broader thing than playful thinking (and deeper and more sacred in some ways), so playful thinking doesn’t at all encompass all of fun. Fun and playful thinking are related though; playful thinking is supposed to be fun, and at least for me, the issue with playful thinking is that the fun is being stifled. So following on your last paragraph, the deeper thing is fun simpliciter.
Another point, only hinted at by the phrase “serious play”, is that the concept of playful thinking is not supposed to imply unseriousness. Seriousness is not the same as explicit-usefulness-justification, because play can be serious but it’s almost impossible for activity driven by explicit-usefulness-justification to be genuine, fully deep fun. (It can be somewhat fun, and some people are blessed to have explicit-usefulness-justifications that spur them into activity that then becomes genuine, fully deep fun. I can sort of do that but not fully, especially because my explicit-usefulness-justifications are pretty demanding and don’t want me getting confused about what counts as success.) Serious play, in its seriousness, can involve instruction and taste. It could involve a mentor giving you harsh feedback. It could involve, for example, you saying to yourself: the thing I’m learning about right now, in the way I’m learning about it, does it access [what intuitively feels like] the living, underlying, hidden structure of the world? And then modifying how you’re engaging to heighten that sense. It could involve your case of learning a mode of thinking from someone else.
> trying to claim it as my own
My two cents (although I’m worried about intruding on this, and worried about other people retroactively intruding such that the process is distorted): if at some point you realize that you’ve gained a lot on claiming it as your own, it would be very valuable to describe that to others. (If you’ll allow a flight of fancy: We can only send messages backwards in time by a few years, so only messages that are taken seriously as a priority to transmit backwards in time will be relayed fast enough to outpace the forward flow of time, and make it back to primordiality.)
Oh yeah, that’s (potentially) a great example. At least in the human regime, it does seem like you can get sets of people relating to each other so that they’re very deeply into conflict frames. I wonder if that can extend to arbitrarily capable / intelligent agents.