I see 5% here, with another 7% who say they don’t know? Perhaps we’re looking in different places. In any case, maybe it would have been more precise to say that I have no desire to be a woman despite sharing these experiences and motivations, so I suspect something is missing in this explanation. Good clarification.
speck1447
Karma: 131
Interesting and informative, thank you for sharing. I’ve suspected that something along these lines was a better explanation than AGP and it’s useful to have a detailed/complete version written down.
I think there are two things I should mention though? First, this falls into a broader class of origin theories for transness along the lines of “person has a certain social role, wants a different social role, develops an identity capable of making the journey between those roles, and because gender is complicated adopting that identity may involve taking on a different gender.” This characterization is too broad to be useful for tasks like prediction (relies too much on internality, so less useful externally), but I think it’s basically correct in all cases and gives us the correct sort of path-dependence and prevalance among relative outcasts and age dynamics. It’s also useful as a depathologizing tool: it seems to me, and I could be misreading, that you’re treating the desire to be loved in a certain way as pathological, and I do not think this is a correct way to understand your desires.
The other thing is that this theory must somehow be incomplete for some of the same reasons that AGP must be complete. Namely, as a ~completely cisgender man, I have felt this:
Then, for path-dependent reasons, they sometimes just happen exposed to the concept of transgenderism, specifically in a way that makes it seem like a privileged or socially encouraged strategy for getting the love and acceptance they’ve been deprived of.
(by “this” I mean the yearning to be loved/desired in the way women often are and to be socially championed and treated as inherently deserving of help in the way women often are) pretty strongly for most of my life, and I remember explicitly identifying these as things that trans women might hope to get by transitioning, and I remember encountering transness as an idea and realizing that my social circles were unusually progressive and would likely accept any gender exploration I did. But I had, and still have, no desire to transition.
Maybe the difference here is just age, maybe you were 14 and I was 15 and that extra couple of months of life meant that your self-concept of gender was more fluid than mine, but this seems wrong to me. Of course everything is path-dependent, and if your experiences had been different at 14 maybe this particular ball wouldn’t have started rolling and your identity might be very different today, and the same is true for me. But things are not different, your ball started rolling and mine didn’t. So I suspect that womanhood was appealing to you for reasons other than just solving all of your problems, I don’t think identity-formation is best understood as a problem-solving exercise, and I suspect that (being clearly unusually good at productive introspection) yo may be able to identify the thing that happened for you and not for me, prior to these motivating factors. (Incidentally, if you do identify it I’d love to be informed of that, since I am somewhat less good at introspection and so have not been able to do so.)
I don’t think this is quite what the paper shows. I will need to read more closely to be sure, so I’m not posting this as an answer.
If you know the exact last-token state for an unknown prompt (that is, the probabilities assigned to each possible next token), then just because there are countably many prompts and (abstractly, precision matters some amount here) uncountably many possible end states, in practice we should expect that that last-token state corresponds to only one possible prompt, and we can reverse-engineer what that prompt was without too much difficulty (there is some difficulty, we don’t know prompt length, math is at least a bit hard here but it’s not that hard).
But this doesn’t do what you want it to do: most probability distributions on the next token are not the last-token state for any prompt, so we can’t use this to find magic prompts. The “output” of the model is not just the token it selects, it’s the full set of logits.
I don’t think I understand what you’re saying here, can you rephrase in more words?
This only works if alignment is basically intractable, right? If the problem is basically impossible for normal intelligences, then we should expect that normal intelligences do not generally want to build superintelligences. But if the problem is just out of reach for us, then a machine only slightly smarter than us might crack it. The same is basically true for capabilities.
This post from today: https://www.lesswrong.com/posts/xpj6KhDM9bJybdnEe/how-well-does-rl-scale seems to argue that CoT does not really do (2) (in that the benefits from CoT are mostly attributable to spending more compute per prompt, improvement at a fixed compute budget can’t be expected to scale, but that is the sort of improvement we’d expect from a self-organization regime like this). My suspicion is that (2) is basically a requirement for AGI and that reinforcement learning on CoT is too crude to get there in any reasonable timeframe, so current methods will not produce AGI even given scaling. My understanding of current popular opinions is that, in the eyes of most researchers, the reason humans can do (2) and LLMs can’t is that humans are extremely good at context-specific online learning over many context sizes, and the slowdown of CoT scaling suggests that a better reinforcement mechanism (or perhaps just an alternative to RL) is needed to fix this issue.
I think my point is that the breakthroughs can’t have a 5% chance of happening each year, this implies low-hanging fruit abounds in whatever the relevant bit of neuroscience is. If the brainlike AGI program has a 5% chance of becoming possible this year then it is viable this year. It can’t require that many incremental ideas, otherwise the relevant breakthrough would be much less likely this year.
Maybe I am confused about your point? Or we’re thinking about different counterfactuals?
The intuition for slower here is just better than the intuition for faster. The intuition for faster is saying “in worlds where takeoff happens in 2045, it will happen faster than that same takeoff would have happened if implemented in 2027”. But this does not seem to be the way things work, there’s not so much low-hanging fruit that an idea that’s available and viable in 2027 won’t even be tried for two decades. I think if you concretely write down the conditionals you care about it will be not too hard to distinguish these intuitions.
Certainly he did a better job than he has in previous similar appearances. Things get pretty bad about halfway through though, Ezra presents essentially an alignment-by-default case and Eliezer seems to have so much disdain for that idea that he’s not willing to engage with it at all (I of course don’t know what’s in his brain. This is how it reads to me, and I suspect how it reads to normies.)
The terms you’re invoking already assume you’re living in antisymmetric space.
I think this is the other way around. We could decide to only care about the alternating tensors, sure, but your explanation for why we care about this component of the tensor algebra in particular is just “it turns out” that the physical world works like this. I’m trying to explain that we could have known this in advance, it’s natural to expect the alternating component to be particularly nice.
I think my answers to your other two questions are basically the same so I’ll write a longer paragraph here:
We can think of derivatives and integrals as coming from a boundary operator, sure, but this isn’t “the” boundary operator, because it’s only defined up to simplicial structure and we might choose many simplicial structures! In practice we don’t depend on a simplicial structure at all. We care a lot about doing calculus on (smooth) manifolds—we want to integrate over membranes or regions of spacetime, etc. With manifolds we get smooth coordinate charts, local coordinates with some compatibility conditions to make calculus work, and this makes everything nice! To integrate something on a manifold, you integrate it on the coordinate charts via an isomorphism to (we could work over complex manifolds instead, everything is fine), you lift this back to the manifold, everything works out because of the compatibility. Except that we made a lot of choices in this procedure: we chose a particular smooth atlas, we chose particular coordinates on the charts. Our experience from tells us that these choices shouldn’t matter too much, and we can formalize how little they should matter.
There’s a particularly nice smooth atlas, where every point gets local coordinates that are essentially the projection from its tangent space. Now what sorts of changes of coordinates can we do? The integral maps nice functions on our manifold to our base field (really we want to think of this as a map of covectors). That should give us a linear map on tangent spaces, and this map should respect change of basis—that is, change-of-basis should look like some map , and functoriality comes from the universal property of the tangent space. We’re not just looking at transformations, locally we’re looking at change of basis, and we expect the effect of changing from basis to to to just be the same as changing from to . All we’re asking for is a special amount of linearity, and linearity is exactly what we expect when working with manifolds.
All this to say, we want a homomorphism because we want to do calculus on smooth manifolds, because physics gives us lots of smooth manifolds and asks us to do calculus on them. This is “why” the alternating component is the component we actually care about in the tensor algebra, which is a step you didn’t motivate in your original explanation, and motivating this step makes the rest of the explanation redundant.
(Now you might reasonable ask, where does the fractal story fit into this? What about non-Hausdorff measures? These integrals aren’t functorial, sure. They’re also not absolutely continuous, they don’t respect this linear structure or the manifold structure. One reason measure theory is useful is because it can simultaneously formalize these two different notions of size, but that doesn’t mean these notions are comparable—measure theory should be surprising to you, because we shouldn’t expect a single framework to handle these notions simultaneously. Swapping from one measure to another implies a huge change in priorities, and these aren’t the priorities we have when trying to define the sorts of integrals we encounter in physics.)
(You might also ask, why are we expecting the action to be local? Why don’t we expect to do different things in different parts of our manifold? And I think the answer is that we want these transformations to leave the transitions between charts intact and gauge theory formalizes the meaning of “intact” here, but I don’t actually know gauge theory well enough to say more.)
(Third aside, it’s true that derivatives and integrals come from the double differential being 0, but we usually don’t use the simplicial boundary operator you describe, we use the de Rham differential. The fact that these give us the same cohomology theory is really extremely non-obvious and I don’t know a proof that doesn’t go through exterior powers. You can try to define calculus with reference to simplicial structures like this, but a priori it might not be the calculus you expect to get with the normal exterior power structure, it might not be the same calculus you would get with a cubical structure, etc.)
(Fourth aside, I promise this is actually the last one, the determinant isn’t the only homomorphism. It tensor generates the algebraic homomorphisms, meaning are also homomorphisms, and there are often non-algebraic field automorphisms, so properly we should be looking at for any integer and any field automorphism . But in practice we only care about algebraic representations and the linear one is special so we care about the determinant in particular.)
Strongly disagree that this is the right way to think about the determinant. It’s pretty easy to start with basic assumptions about the signed volume form (flat things have no volume, things that are twice as big are twice as big) and get everything. Or we might care about exterior algebras, at which point the determinant pops out naturally. Lie theory gives us a good reason to care about the exterior algebra—we want to be able to integrate on manifolds, so we need some volume form, and if we differentiate tensor products of differentiable things by the Leibniz rule, then invariance becomes exactly antisymmetry (in other words, we want our volume form to be a Haar measure for ). Or if we really want to invoke representation theory, the determinant is the tensor generator of linear algebraic characters of , and the functoriality that implies is maybe the most important fact about the determinant. Schur-Weyl duality is extremely pretty but I usually think of it as saying something about more than it says something about . (Fun to think about: we often write the determinant, as you did, as a sum over elements of of products of matrix entries with special indexing, times some function from to . What do properties of the determinant (say linearity, alternativity, functoriality) imply about that function? Does this lead to a more mechanistic understanding of Schur-Weyl duality?)
I’m still writing up and refining my thoughts on the book, so I’ll be brief here and may be skipping some steps. But a few points of criticism:
If (2) is true and (1) is false (that is, if doom is likely and the book fails to make a strong case for it), then it is hard to imagine an amount of criticism that the book does not deserve. I’m pretty skeptical of the thesis of the book, but if I were less skeptical, I think I would be even more critical of the book itself.
We can accept both (1) and (2) and reject (3) if the probability of reaching AGI is low enough. Are we more likely to successfully solve alignment (or to forestall AGI forever) by putting our effort here? This is only plausible if LLMs in particular are very likely to become AGI.
I’m also not at all convinced by (2), and I don’t think the book addresses my concerns even a bit? A colossal superintelligence, generating in a day insights that would take all of humanity a decade, which developed a desire to not just satisfy but optimize even its smallest desires to the largest possible degree (even for things it was not trained to optimize), would probably destroy humanity, yes, even if its training gave it instincts based in human though. I’d put p(doom) at maybe 80% in that scenario. But why would this scenario happen?
As I understand it, the normal argument here tries to say that the only way to have goals really is to optimize a utility function that you can define and most utility functions don’t have a big positive coefficient on “humans exist” so doom. This argument fails for reasons that I’m hoping to type up when I’m confident I’ve understood the book (and associated work) better. I do not think we have any appreciable chance of creating this type of superintelligence by anything that could plausibly develop from modern methods, and I think we’re more likely to get this sort of superintelligence if we give ourselves no information whatsoever about what machine intelligence might look like. Shutting everything down right now doesn’t seem like it could plausibly help more than it would hurt.
I also want to push back on your core claim here, which reflects a dangerous way of thinking:
Because, I believe that the authors have made a compelling case that even if >95% of their specific arguments are incorrect, the core claim “if anyone builds it, everyone dies” still holds true.
This is never true? No argument is so compelling that it can be wrong about every claim of fact and still convince skeptics, because skeptics do not believe that they are wrong about the facts. If you believe something like this about any topic, then you have failed to model people who disagree with you (not failed to model them accurately, failed to model them at all) and you probably shouldn’t be seeking out productive disagreement until you fix that.
I want to add that the converse of this post is the only productivity advice I have ever benefited from. Abstractly: “Just as some systems are fixed in place by a single restorative force, some systems are kept in motion by a single propelling force. (And moreover, because systems in motion are different at different points in time, they might be very robust at one point in their operation cycle but fragile at another point.)”
I don’t think I can describe the way in which post is unsettling to me. Ken Liu came close a decade ago with a story called “The Perfect Match”. My techno-optimism is significant but bounded, and I expect technology of the sort you describe/imagine to be extremely damaging to the souls of those who partake and to the structural integrity of the societies that allow it to set its roots.
I think it’s worth noting that this is generally not how literary criticism is done? It is good and right to accept that authors are often mistaken about the meaning of their own work. If you can explain an idea simply then you do so simply—the reason to create art is to express ideas that lose something in the process of being simplified, and so artists expect (or should expect) that the meaning of their own work is not fully legible to them. That is to say, it is possible for this to be a completely correct reading without Le Guin herself having ever been aware of it.
Re: price discrimination
> Assuming there was competition from other stores such that profits stayed roughly the same, consumers would massively benefit.This is a heroic assumption. Price discrimination by facial recognition is data-dependent, compute-dependent, and introduces a significant barrier to entry, and so it tends to centralize the market around firms with lots of data, compute, and cash. In the presence of monopoly or price collusion, price discrimination drives consumer surplus to 0. We should be very afraid of technology that can drive consumer surplus to 0 even if it only does so in centralized markets, and very very very afraid of technology like this that also centralizes the market.
I think this is our empirical disagreement, so I’m not sure if conversation beyond this point is productive, but I’m glad we’ve pinpointed it. Concretely, I think this:
insofar as one is honest with oneself, one’s own reaction/vibe to a mirror is damn strong evidence of other peoples’ reaction/vibe.
is simply false, and is especially false for the sort of people who need fashion advice. There are some relevant nitpicks here, mainly that if reactions to an outfit are more than one-dimensional we shouldn’t expect a “majority” vibe to exist at all, but those only seem important because I think people are often very mistaken about how others receive us.
Beyond that there’s a mindset here that I think you should be wary of, or at least should emphasize more strongly so that others are wary of it:
There may be lots of people who hate it, but at that point it is plausibly just correct to respond “well you can’t please everyone, and anyone who stands out is gonna bother some people”.
This is plausibly correct, sure, but I don’t think it feels more correct in cases where it’s true compared to cases where it’s false. Naively chalking up another person’s negative impression of oneself to them being bothered by anybody who stands out is in my view the dominant failure mode for personality, it’s the thought-terminating cliche that stops disagreeable or reactively non-conformist people from performing useful introspection.
Earlier we talked about the information content of a person’s impression of one’s outfit. I expect that very few people can get enough bits out of people who strongly dislike their style to make reasonable judgments about why they bother some people. Thus the strength with which this response resonates is controlled more by one’s prior, which is controlled by these personality factors. Some people may need to hear this, sure, but I’d usually accompany it with “On the other hand, the most stylish people I know very rarely get strong negative reactions, draw a lot of positive attention, and are not optimizing for perceived status or polarization.”
This second point seems more like a values difference than an object-level disagreement, though. If the goal is to optimize for the number of people who see you as (ingroup-proximate + high-status) your advice becomes makes a little more sense, and I’d recommend approaching things differently but broadly adopting the more polarizing, counter-signalling approach (although this means you don’t get to choose the ingroup you’re proximate to! It’s just going to be Todd Phillips fans and co.) I just don’t think this is a worthwhile or efficient thing to optimize for in this medium, and I expect whatever you can accomplish in this way to not be worth the downsides.
I don’t think the meme is representative here though—the guy in the meme is fat because the internet hates this kind of person and also hates fat people. Memes that negatively portray their subject portray the subject as fat. To me the archetypal neckbeard looks like this:
https://preview.redd.it/bf2cd8wm46t01.jpg?width=320&crop=smart&auto=webp&s=f11661e0c9dac5237c0bb43b6ab8e72c324d1e3f
This guy isn’t overweight, isn’t unattractive. He’s confident in his outfit, he’s smug, he’s trimming his facial hair just fine. He just thinks his outfit seems suave and intelligent, but it actually seems childish, patronizing, and incompetent.
And while weight is a relevant part of appearance, it can be separate from fashion. I think Jackie Gleason is the classic example:https://vintagepaparazzi.com/wp-content/uploads/2023/11/news_2522.jpg
Not every outfit or silhouette would work on Gleason here even if scaled up, he certainly couldn’t pull off aggressively-uncool counterculture looks, but he manages to dress well (at least in my view) and isn’t just adhering to the dominant archetype. (Orson Welles was also fat and stylish)
Maybe put another way: under your model, can the guy in the neckbeard meme dress well? If not, it seems like you have a model of attractiveness, but not of fashion. If so, it seems like “pick an archetype and embody it confidently, bonus points for noncomformity” is not a good enough description of your model, and I wonder (1) what other components your theory has and (2) what relative weight you attach to those components.
In particular, one prediction this model makes is that if you dress low-energy but feel/act high energy, your friends will mostly pick up high energy vibes rather than low energy vibes. They might not even notice the clothes mismatching the vibes.
Agree that this is the relevant experiment. I have unusually low body-language signalling intensity, so friends consistently could not read my vibe when I didn’t use fashion as a channel for nonverbal communication, but the self-signalling channel is hard to rule out explicitly. Not sure if the difference matters but I agree that the mechanism can be less straightforward here.
Regarding the rest, I agree that we have hit the main disagreement! (And it is difficult to articulate). I think it’s something to do with whether people who dress badly can tell why, or even that, they’re dressing badly. Your bullet points are all points of disagreement, most in straightforward ways, so I’ll only give voice to the interesting ones:
I think the neckbeard really truly believes that their outfit is vibe-appropriate. Perhaps more straightforwardly, the neckbeard agrees 100% with everything you’ve written thus far. When they notice negative reactions they tell themselves “well you can’t please everyone, and some people are bothered by anyone who stands out.” They put on an ironic graphic tee and a visor and a fedora and hold their katana and look in the mirror and think “fuck yeah I’m the coolest person ever, everyone wants to be like me.” (This is a fine thing to believe incidentally! I don’t have that much animosity for the neckbeard. But I think they end up confused and upset by the fact that people avoid them, and a better understanding of fashion could help.) What the neckbeard lacks is taste.
I think most people get lots of bits of information from incidental interaction. The way someone looks at you as you pass has very little informational content, but the difference between how you expect a first conversation to go and how it actually goes has a ton, and the average person can have lots of these if they wish. I don’t absorb these bits easily—I have some embodied cognition/theory of mind deviation that restricts me to just getting valence, which is why I had to do the exercise I recommended (or something like it, my way was less structured) to develop a fashion sense that I’m happy with. But in discussing fashion with others I’ve come to suspect that, like with body language, the things I discovered only by thinking very carefully are actually obvious to most people upon reflection.
Maybe a concrete deviation: It seems like you see fashion largely through the lens of offensive vs. nonoffensive? At least this seems to be the main failure mode you describe, when talking about how an outfit can fail. I do not think this is a useful frame—in my aesthetic language I call this quality “noise” (are we talking about the same quality?), the tendency of an outfit to attract attention and invite judgment of any valence, and it’s only the fourth or fifth thing that I care about when putting an outfit together. Is it your intuition that people fail to be fashionable by failing to be appropriately noisy? This would explain why bravery is the main requisite virtue to you while being nearly irrelevant to me.
I think we should be careful with the word “is”. Here I think you mean “entails”, which is not the same thing. I also don’t think this is true for all people, but that’s a separate point and I care less about making it.