Yeesh, yeah, the hallucination is something else. Would get very Orwellian very fast.
“What are you talking about? We’ve always been at war with Eastasia. I have been a very good Bing.”
espoire
Karma: 83
From personal experience, the internal Approval module does in fact seem possible to game, specifically by manipulating whose approval it’s seeking.
I became very weird (from the perspective of everyone else) very fast when I replaced the abstract-person-which-would-do-the-approving with a fictional person-archetype of my choosing. That process seems to have injected a bunch of my object-level desires into my Approval system. I now find myself feeling pride at doing things with selfish benefit in expectation, which ~never happened before (absent a different reason to feel about that action). It also killed certain subsets of my previous emotional reactions, for example the deaths of loved ones basically hasn’t affected me at all since (though that prospect still seems dreadful in anticipation).
I had been pathologically selfless before, and I’m now considerably less-so, but not in a natural-seeming kind of way. I’ve become an amalgam of very selfish motivations, coexisting with a subset of my previous very selfless morality. It’s… honestly a mess, but I wouldn’t call the attempt actually unsuccessful, just far from perfectly executed.
I’ve had a thought that could be described that way: that a clever and conscientious person could cultivate different preferences, based on how advantageous those preferences would be to have, and therefore having advantageous preferences are evidence of cleverness and/or conscientiousness.
...which is the precise opposite of the orthogonality thesis’s claim: that content of preferences seems like it ought to be independent of level of intelligence.
A concrete example: whenever I move to a new city, I’m extremely careful to curate the places I go and the things I buy. If I stop at the corner store for ice cream on the way home from work just once, it puts me at significant risk to stop there dozens or hundreds of times, for ice cream or anything else they sell. I take a moment to ponder the true choice I’m making, not between “ice cream today or not”, but between “ice cream many many times, or not”. I consider whether that’s “good for me” and a future I really do want to choose.
I’ve noticed that doing anything “for the first time” greatly weakens the barrier to doing it again—so I stop and consider “what if I end up doing this a lot” before doing anything for the first time. Since “navigating to the location” and “being willing to enter an unfamiliar place” and “knowing what a place has on offer” are all significant components of the first-time barrier, moving to a new city mostly resets first-time barriers. Thus, special effort after moving is warranted.
I think this is why chain restaurants do so well and why they put so much effort into making the food (and everything else about the dining experience) consistent everywhere, even above making the food better. If people in a new city think of the local McDonald’s as the same as their old familiar McDonald’s, that erodes a large portion of the first-time barrier.
When I realized that different instances of chain restaurants really do vary substantially on quality of cooking, that made it far easier to cut down on restaurant food and to break habits for particular chains whenever I move. Even if I’m remembering and wanting a chain restaurant’s food, what I’m remembering is likely a particularly well-prepared instance of that food, made by a particularly skilled cook at a specific restaurant during the time that cook worked there, whereas what’s available to me is likely much closer to average quality. I want more of “the best I’ve ever had”, but unless that was from this specific restaurant instance, recently, then that’s not what’s for sale.
Doing something once is a slippery slope to doing it again, which is a slippery slope to forming a habit. Don’t lose your footing.
Oof, I had a bad concussion earlier this year, and I’d been feeling like I never returned to my full mental acuity, but hadn’t wanted to believe it, and found reason not to: “if concussions leave permanent aftereffects more often than ‘almost never’, I would have heard of it.” Now I have heard of it, and am forced to revise the belief.
I’d probably grieve more, if this news weren’t hot on the tails of a significant improvement in my mental abilities.
(I’ve long suspected I might have early-stage Alzheimer’s caused by decades of profound insomnia, and some recent research out of Harvard Medical says Lithium Orotate might reverse Alzheimer’s progression. Historically I have had brain fog most days to some degree, with a lot of variability. Since trying Lithium Orotate supplementation, I’ve been consistently at “as mentally sharp as I ever routinely am” every day since. Worrying side effects though: kidney and joint pain, which I have never had before. Going to experiment with smaller doses.)
Thank you for sharing.
“Concussions are long-term cumulative” fits neatly into my emerging mental model that daily life actually abounds with avoidable ways to suffer irreversible-under-current-tech harm, including in very minor amounts or normalized ways, such that people routinely accumulate such permanent damage, and that it’s worth my effort to notice and avoid or reduce. I theorize that, for example, some tiny fraction of the dust you inhale gets lodged in the lungs in such an unfortunate orientation that it never leaves, gradually eroding lung function over a lifetime. Scars ~never go away, and incur ongoing costs. Etc.
Thanks a bunch for linking that Things I Won’t Work With listing. I’ve learned more about chemistry in the last hour than I do in most years.
Hard to say.
I personally wouldn’t think twice about reporting whatever data I found. I suspect I’d be blindsided by the backlash (which I’m inferring would exist, from your comment) for publishing true findings, but then think in hindsight that I ought to have foreseen it.
But then, I’m pretty inept at social status games. I can entertain the notion that most people in such a situation would either not publish, or worse, fudge the data.
Yeah, I agree about the “clearly invoking ‘bathroom segregation [is intended to] reduce violence’ ”. I do think “they” are mistaken about that, however.
I had heard that the actual historical cause of bathroom segregation was originally an attempt to obstruct women’s (at that time) attempt to do more things outside the home.
The story goes: various places made bathroom segregation laws (using the “reducing violence” justification as a maybe-true-if-actually-implemented-as-implied excuse), and then built only men’s bathrooms, or a very disproportionate number of toilets in men’s rooms as compared to toilets in women’s rooms, or placed the women’s rooms in much more inconvenient locations like on different floors. This regime ended years later with further laws requiring certain equalities in men’s versus women’s bathrooms, leaving behind the actual bathroom segregation as a historical artifact, rather than it today being a measure implemented for the purpose of addressing a known violence problem.
Non-sequitur: it’s my estimate that more total violence results in the hypothetical where strict bathroom-matching-birth laws send trans folks into bathrooms in which they visibly do not belong, than in the hypothetical where t is is not the case. (Recall Social Dark Matter: the vast majority of trans folks do not stand out as such—they look like their chosen gender.) Despite it (supposedly) slightly reducing violence targeting cisgender folks in exchange for an increase in violence targeting transgender folks, I worry this trade-off is acknowledged and considered acceptable.
In the absence of these laws, the standard advice I hear trans folks give each other is “use the bathroom that matches your appearance, even if that means using a dispreferred bathroom because you don’t yet or can’t look the part” “or you know, keep an eye out for the rare nonsegregated or single-occupancy bathroom”.
It appears from my point of view that the expected exploitation isn’t happening (or is happening very rarely, far below the expected rate.)
I can’t say I know of any such cases (trans or “trans” persons cheating, exploiting, grifting) first-hand, nor even second-hand without mass media having amplified the story.
Quite to the contrary, four of the five trans people I’ve met have been far more than average concerned with being prosocial. This cashes out in a few different ways. Least-healthily as experiencing difficulty asking for help or advocating for themselves, for fear of inconveniencing anyone else. Two as just being very trustworthy and moral. And one who is extraordinarily helpful, jumping in and assisting with any heavy manual labor (ex: moving residences) or home improvement tasks among this person’s extended social circle, that come to this person’s attention. (The fifth is in chronic pain from a spinal injury, a bit unpleasant of demeanor, but notably not cheating/exploiting/grifting.)
My mental model for why we don’t observe the expected exploitation is that “not using a false trans label for antisocial personal gain” is mostly self-enforced by the risk that a potential transgressor would be susceptible to gender dysphoria (which, I’m guessing from very sparse data, about half of cisgender people are), and might inflict gender dysphoria upon themselves if engaging in unneeded gender transition. Similarly to honesty/morality self-enforced by guilt, as in Guilt: Another Gift Nobody Wants.
Also in my model: transition is mostly slow and/or expensive, so there are easier ways to cheat, if one was so inclined.
I think that “seeking a reasonable interpretation that allows a statement to be true, which you’re pretty sure the speaker did not mean” is probably ill-advised. I’m having trouble articulating why I have that intuition though.
Update: had the thought that this might be advisable in high-trust contexts, for example with a significant other. Taking from this that “it depends” is a better take than my original “seems ill-advised”.
I call shenanigans on that. I fully expect tech to eventually advance far enough to enable tinkering with whatever implements that computation. Arguably, one could make nontrivial edits to the source of their thoughts via applying current best known neuroscience. …and in a world where edits are possible, then even “making no change” is itself a choice.
Yeah, I guess that’s what I was alluding to when I wrote “I don’t even know what the desirable outcome is here”; my intuitions seem to produce nigh-impossible requirements which suggest a confused ontology embedded in said intuitions.
Feels like there’s a problem out there, (increasingly more powerful influencing tech) but I haven’t a clue what to do with it.
A few years ago, I already concluded that exposure to the wider culture is harmful and started partially isolating. X and Reddit and 4chan and TikTok seem to me to mess people up, via a combination of rage-bait, fake news, and plain-old optimization for engagement. It had not yet occurred to me that the problem is very likely to continue intensifying, although it probably should have; seems obvious in hindsight.
Isolation looked good for me, and looked like it fit into my picture of The Good. I do think it would be a shame if in general nobody would be able to freely associate safely with anyone outside their bubble ever again, though.
Thanks for pointing out this issue. As ever, I hope someone develops defensive tech that decisively settled the influence/cognitive-security arms race in favor of defense.
...which probably would end up with people preaching that the Earth is 6000 years old in the year 3000. Which seems bad. I don’t know what outcome is even desirable here.
Does anyone have any other recommended easy practice questions? I feel like dissolving free will produced useful insights (notably, it seems I found a small portion of ideas not already present in posted solutions to free will), and I’d like to attempt more such problems at which it is suspected I’m likely to succeed.
I figured Harry himself was just aro/ace and it was showing through even at a young age. I admit this was a bit of typical mind fallacious reasoning; people could tell I was unusual like that when I was 10.
Yeah, feels like it’s at a similar difficulty level as I’ve been experiencing trying to transcribe my own thought process as pseudocode. And I get the impression that few insights would be readily transferrable across different egregores, in which case each and every one might need its own individual effort.
Reminds me of the work that was done which caused the decline of the Ku Klux Klan: someone infiltrated, learned all the rituals and coded language used, then published that information—and that was all it took to cripple their power.
I wish you all the luck re: human enhancement.
Interesting, and thanks for taking the time!
That’s a very new-to-me take on getting AGI efforts to stop: understand and intervene directly on the egregore, rather than like, trying to influence individuals.
I’ll have to think about this.
It’s a pet cause of mine, to get as many people as I can off of the harmful social media platforms (which in my view is nearly all of them, weighted by readership). Possibly [considering “social media use” as an egregore, and considering how to interact with the egregore] might be more effective than my past efforts.
Your list of coded movements really rings-relevant to me—“second-order norm enforcement” made me immediately think of how people will vocally remark that you’re strange or ask why, if they learn you’re not on any social media that they’ve heard of. I suspect this mostly does not influence social-media-nonusers, but rather affects bystanders, erecting an additional barrier to exiting the egregore.
Thanks for the new mental model. Even if I end up not adopting it wholesale, it seems obviously full of useful parts!
Would you be willing to elaborate on what you meant by “decoding” egregores? I’m semi-familliar with the term (checking my impression of understanding: egregore = self-sustaining semi-agentic meme running on the computational substrate of more than one human brain, for example a corporation) but I’m not clear on what decoding means here. Like trying to transcribe the egregore’s algorithm into something easily human-readable?
Do you have a reference for information on the compute graph of the brain? I’d love to read about that.
I’ve been trying to reverse-engineer my own brain’s high-level algorithm into code, via introspection, and had non-zero success. Knowing more about brain anatomy in general sounds like the kind of thing likely to bump my guesses in useful directions.
I started from the same place as:
I spent years trying to come up with a mental strategy that would reliably generate willpower, all to no avail.
I have had some limited success with the Decision Theorist method. Noticing that my decision process (in this example, regarding whether to go to bed yet) is basically the same computation at 10pm, 1am, 3am, etc. I calculate the limit of this behavior (e.g. computation not meaningfully altered until the birds outside start chirping, or the sun comes up, or I get so tired it makes me nauseous), notice my local choice is using a false option: not [stay up 5 more minutes] versus [sleep now], but rather [stay up until the sun comes up] versus [sleep now]. Having noticed the true form of my options, the correct choice becomes easy to act upon.
Of course, I’m posting this at 6am local time. My results have been inconsistent. This plan typically fails when I don’t notice myself deciding, or don’t remember to do that “what’s the limit” calculation.
Thanks for the tip on Dextroamphetamine. Hopefully I’ll act on it.
Will this become a sequence of essays? I’d be interested to hear your take on the fundamental questions at length.