Discord: LemonUniverse (.lemonuniverse). Reddit: u/Smack-works. About my situation: here.
I wrote some bad posts before 2024 because I was very uncertain how the events may develop.
I do philosophical/conceptual research, have no mathematical or programming skills. But I do know a bunch of mathematical and computer science concepts.
I think for this post to be more substantial, it would need to
Explain why you don’t compare AF to analytic philosophy. Seems like an overall better comparison. You can criticize analytical philosophy for being disconnected from reality too. Your post doesn’t explain why you go for continental philosophy.
Provide better examples. The Critch one seems a bit non-sequitur (“AF is too vague… no, now it’s too formal!”). You need to actually criticize his work to make your point, instead of just asking “is his work good… or is it bad?” Nate’s post about the sharp-left turn is not an example of constructive AF research. It’s a valid example, it just shouldn’t be central to your point.
Have real conclusions. What should AF researchers actually do? Abandon stuff like decision theory entirely? Do it better in some specific ways? Then you could write about the specifics.
I believe this is partly due to the specific writing style of Eliezer Yudkowsky, which tends to be verbose and prone to metaphor and other narrative approaches of argumentation.
I didn’t study EY’s other technical writing, but his Arbital sequence is very concise and narrative-free. For example, check out the article on ontology identificaion.
A lot of commentary on AI risk and alignment theory takes place on LessWrong. As a community blog intended to provoke new directions of thought, LessWrong is fantastic! An important first step in developing new theories is to work on concepts, while formalization and evidence come later. But a lot of actual alignment work also occurs directly on LessWrong, or the associated AI Alignment Forum. Because of this dual use, I feel that there has been a mode collapse, where it is difficult to distinguish between metaphorical/conceptual work and rigorous research.
I think there are factors which make rigorous and informal work hard to tell apart (for example, see the historical and ahistorical debates about the rigour of early calculus). But those factors have nothing to do with forum norms and you don’t get into them. Maybe you mean “it’s hard for an outsider to understand which work has to be taken fully seriously” (but you don’t seem to really talk about that either) or “researchers disagree which things are rigorous” (but that has nothing to do with forum norms, again).
Btw, it rarely feels like research really happens on LW or Alignment Forum. It seems like research happens in cliques of people and the results get published on forums. But I have little experience here.
Shove your insults up your own ass, mate.
This is a cultured place, why the gracious obscenity? If that’s a part of the kink, I suggest putting it into a collapsible section.
(didn’t read the post, just saw this shit in my feed)
I’m confused by this post (and why a perfectly fine response by williawa was downvoted). The post feels like a fully general argument against Bayesianism, probabilities, and reasoning (about novel situations) in general. Judging by the author’s clarifying comment, it is a fully general argument:
Maybe I am advocating for a stance of Knightian uncertainty or maybe just saying that the standard Bayesian approach can plausibly lead you to be much too overconfident in your views. I think reasoning about ASI a priori doesn’t makes sense in the same way that the question posed in Bertrand’s paradox doesn’t make sense.
But then the post shouldn’t have been about counting arguments specifically!
Also, a nitpick:
I wanted to write this up because I think that a lot of disagreements about alignment bottom out into differences in projections, but people often argue as if they’re disagreeing about the territory.
People do disagree about the territory though?? The claim that people’s disagreement is merely semantic is shocking, it requires examples.
Type of guy obsessed with guys here. (Enter my shameless plug.)
Are you interested in fictional characters? Harry Potter, Star Wars, LOTR, ATLA, MCU, anime, etc? I have a system which is the easiest to illustrate with fictional characters, but I believe it applies to real people too.[1]
Biggest downsides:
I don’t know how to formalize the system to make it falsifiable. The system is partially undefined even from an informal POV.
There are countless characters and people not fitting any type I know. Though the general methodology of my system can be applied anyway.
Getting into the system requires some time commitment.
The biggest upside:
The system is philosophically interesting. I believe it’s not similar to any other classification or any other model (of anything).
Types are “absolutely different”, “different in every property”. Like Bouba and Kiki. Or like Indra’s net, if every jewel was unique.
Each type maximizes a unique theme along different dimensions. Each type also maximizes unique sub-themes (related to the main theme) along different dimensions.
The dimensions in question are usually those:
A. Life trajectory, fate.
B. Personal relationships. (Loved ones, family, close friends.)
C. Social status.
CA. Physical status. (Physical powers, physical traumas, disabilities.)
D. Psychology.
E. Outward attitude.
Themes and sub-themes are defined over lower-level facts about characters. When analyzing a character, you should first focus on the most abstract and unique facts about them. Facts which are realized in maximally radical, crisp, objective ways.
...That’s all, that’s the entire methodology. Keep in mind that themes are often too vague without sub-themes. And sub-themes are often too vague too if they are not maximized across different dimensions. Also, vague (sub)themes become less vague when you pay attention to the strongest ways to implement them.
Most classifications are “combinatorial” (any combination of some basic features defines a type) or “geometric” (there’s some relatively simple space and each point / region defines a type).[2] My classification is neither. That’s one of the main reasons I consider it very unusual.
I’ll give a couple of examples. 3 types and 50+ characters. Types of Snape, Draco and Sirius.
One theme (fragility) and 3 sub-themes (irreversibility, walling off, dramatism/sentimentality). 19+ examples.
Fragility
Theme of the first archetype is fragility (stubbornness, instability, internal tension, irreversible changes). Often “fragility” is used to describe things which are very easy to break, but I’m talking about very tough things which would rather break than bend. A perfect example of this theme would have the following properties:
A. Irreversible events which change the character forever.
B. Personal relationships filled with death, trauma, guilt.
C. Unstable social status. Isolation despite great connections.
CA. Physical vulnerability despite great power.
D. Great mental strength mixed with crashouts, shocks, traumas.
E. Cold, harsh attitude, mixed with sentimentality. Stoic but easily triggered.
In other words, fragile life, fragile relationships, fragile social and “physical” status, fragile mentality, fragile attitude.
Examples:
Severus Snape. A and B. Guilty in Lily’s death. That radically changed his life. C. Works as a triple agent, which makes his social position very fragile (either side can start seeing him as the enemy). He’s one of the most well-connected yet the most reclusive HP characters. CA. Very powerful, but under the thumb of way more powerful Voldemort and Dumbledore. D and E. Harsh and cold, but soft when it comes to Lily. Calm but petty, Harry and Sirius can easily trigger him.
Kakashi from Naruto. A and B. Sorta guilty in his friend’s (Obito) death. Accidentally killed his remaining friend (Rin killed herself by jumping in the way) which he promised Obito to protect. He never got over this. 49 seconds clip. Before all this, his famous father died by suicide after getting dishonoured. C. A famous elite ninja, but a loner. CA. IIRC Kakashi is pretty unique in terms of ending up in physically vulnerable situations, he’s strong but is constantly pitted against much stronger opponents. One time he got tortured for 72 hours inside an illusion. E. Both harsh and soft.
Elsa from Frozen. A and B. Almost killed her sister Anna. That forced her to live in isolation and be distant from Anna. C. Was hiding her abilities to not lose her royal status / become a monster in the eyes of people. “Make one wrong move and everyone will know.” CA. Despite her godlike powers she was captured and jailed. E. Cold and warm.
Lelouch from CodeGeass. B. Accidentally and deliberately killed many people and blamed himself for it. His relationships are filled with death, trauma and guilt. For example: Lelouch and Euphemia part 1 (~1 minute), part 2 (2:09 − 4:18, ~2 minutes), Lelouch and Shirley (~2 minutes). C. Due to double identity and secrets his strong social status (royalty / leader of a terrorist group) is fragile. D and E. Cold and sentimental. Mentally strong and crashing out.
Scrooge from DuckTales 2017. A. Blamed himself for the presumed death of his niece Della. Scrooge’s past (~5 minutes). B. Lost connection to his family after that. C. Despite being the richest duck, super well-connected, he’s reclusive. Scrooge after the kids leave again (1 minute is enough). E. Harsh and soft. Easily triggered. His average behavior (2 minutes is enough).
Amon from Legend of Korra. A and B. Went through abuse as a child, that changed him forever and pushed him to separate from his brother. C. He’s a leader of a terrorist group, but his position is unstable because he’s lying to them. D. Brutal, but deep down there’s a soft spot for his brother. Amon’s backstory (till 7:21), Amon reunites with his brother (~1 minute).
Jeong Jeong from ATLA. “Once an honored admiral of the Fire Nation Navy, he became disillusioned with his nation and firebending as a whole and defected from the military. He relocated to the Earth Kingdom, where he encountered a small tribe who became his travel companions.” He lost his status, isolated and got psychologically broken a bit. “Fire brings only destruction and pain. It forces those of us burdened with its care to walk a razor’s edge between humanity and savagery. Eventually, we are torn apart.” Calm, but irritable. 2 minutes clip
Nicholas Van Orton from The Game 1997. B. His personal relationships are not doing great… father died by suicide, brother and wife are estranged. C. Rich but reclusive. About to lose everything. D and E. Cold but vulnerable. Traumatized.
Rust Cohle from True Detective. “Character overview” describes it better than I can. A. After his daughter was hit by a car and killed, his life spiraled out of control and he became deeply cynical. B. His relationship with Marty is rocky. “You total shit. You know, this is part on you, you know? You can’t just slide into my life, creating tensions, judging me...” C. Gifted, connected to many people (just due to his job), but isolated. D and E. Tough, but vulnerable and troubled. Rust scenes.
Darth Vader from SW. A. He thought he killed his wife and was severely mutilated. Which completed his descent to the dark side. C and CA. He’s second in authority and power to Palpatine, but Palpatine could easily dispose of him (which Palps did plan to do). D. He kept a soft spot for Luke, that’s why he sacrifices himself at the end.
Gandalf from LOTR. C. Very well-connected, but also a lonely homeless wanderer. His political power is limited because he needs people to make good decisions out of their own free will, that’s his secret mission from God. Faces defiance and reluctance. CA. Divine and powerful, but physically vulnerable and gets hurt a lot (by Saruman, Barlog). E. Both harsh and soft. Super stressed out. Easily triggered.
Doctor Strange from MCU. A. Abruptly loses his career after a car crash injury. C. Very well-connected but isolated. E. Both cold and soft. Calm, but irritable. tony and dr strange arguing for 2 minutes straight.
Goblin Slayer. A. Sole survivor of a goblin attack on his village, saw his sister raped and killed by goblins. C. Has many connections, but is sort of a loner. CA. He’s a goblin killing machine, but he’s only human with no special powers. His job is extremely risky and he goes all out every time, gets hurt often. D. His mind is broken but rigid, killing goblins is his entire identity. E. Cold, but polite and kind. Versus goblin champion (5 minutes is enough), he and Cow Girl (2 minutes), asks for help (6 minutes), cries (2 minutes).
Cigarette Smoking Man from the earlier X-Files. C. Extremely powerful, yet his status is shaky. Often does the dirty work himself, walks around without bodyguards and works with equally powerful individuals (the Syndicate). Very lonely. D. Evil, but cares about Mulder and Scully… sometimes, sort of. E. Rarely emotes, yet many things touch him emotionally. Musings of a Cigarette Smoking Man shows different sides of him. Xmas job (5 minutes), Xmas meeting (3 minutes), episode ending (5 minutes).
Zeno from Hunter x Hunter. C. Powerful, but old and has a very risky job (assasinations). D and E. A single accidental murder can hurt his big pride (he’s a veteran assassin in a family of assassins). Harsh, but not entirely. Him fighting (4 minutes), a civillian gets harmed (2 minutes), gets bothered (2 minutes).
Sir Leigh Teabing from The Da Vinci Code. HEAVY SPOILERS. A. Wants to destroy the Church, free humanity of dogma. C. Wealthy academic, but gets criminal. D and E. Dramatic. Kind/nice but evil/rutheless. Another antihero revolutionary. Greatest coverup in human history.
Patroclus from Hades 2020. B. Feels betrayed by his lover. “He ever tell you of the war in which we fought? I fought, I should say. He could have ended it. But he refused to take up arms. (...) He shared our cause! So what if it was not completely his? If not for his stubbornness, I… well… I would have come here just a bit later, I think.” (c.) C. Lives in Elysium (paradise), but sits alone by the Lethe. D. Really devastated. E. Harsh but soft. patrochilles dialogue & story.
Peter Capaldi’s Doctor (12th) from Doctor Who. C. Sorta like Gandalf—powerful, but vulnerable and depending on others. D. Troubled, hates himself. E. Harsh but soft. Easily agitated. War Speech (4 minutes), Go to Hell (2 minutes), Where I Fall (4 minutes).
Real life example:
Bobby Fischer. A. Became the World Chess Champion, then stopped playing official chess for 20 years. Isolated himself and went mad. C. Before his downfall, was a lonely superstar. After his downfall, became a recluse with good connections. Many were helping him (e.g. Polgar family, Leko, Spassky, Lombardi). He got in trouble with the US government, got detained in Japan, almost deported to the US, but granted a citizenship in Iceland. D. He was very tough mentally (to fight alone against the world), but very troubled and irritable. “Fischer fits ideologically into the context of the Cold War era: a lone American genius challenges the Soviet chess machine and defeats it.” (c.) Kasparov E. He was known to be cold and rude in general, but nice to some in private. For example, in 1962 Tal fell ill during a tournament and Fischer was the only player who visited him.
Vladimir Mayakovsky. B. Had turbulent personal relationships. C. Both a useful and a problematic figure for the soviets. D. Was tough, but troubled. Died by suicide. E. Harsh, but sensitive.
Isaac Newton. Maybe. B. Father died, abandoned by mother, disliked stepfather. C. Powerful but reclusive. Abused his power in personal conflicts. C. Used disguise to find and prosecute counterfeiters (many where killed). CA. Prioritized work over health, D. Suffered a nervous breakdown, blamed friends for a conspiracy. E. Secretive, neurotic, paranoid, vindictive.
Semi-examples:
Carl Fredricksen from Up.
Waternoose from Monsters Inc. B. His friendship with Sally turned out to be very fragile. C. His social status is great. CEO of the company which was in his family for 3 generations. But declining energy production and publicity problems threaten everything. E. Friendly and harsh.
??? from Disco Elysium. Very unique game, I recommend at least watching What Was Disco Elysium Actually About? HEAVY SPOILERS. A. Deserted his comrades. All killed. B. Unrequited love. C. Isolated from the city, but always watching. CA. Fit but frail. D. Decades of survivor’s guilt, shame, bitterness. Still capable of “love”. Another “antihero” revolutionary.
See below. Carl is not a drama-queen (the only grandiose thing about him is the flying house). Waternoose—not enough drama and reclusiveness.
Irreversibility
Fragile!pain is about irreversible consequences.
A. The source of the pain is concentrated in the past, in a single irreversible event or a couple of specific irreversible events.
B. It irreversibly changes personal relationships.
C. It irreversibly changes character’s social status.
D. Character dwells on the tragedy instead of trying to move on.
E. Character’s attitude is strongly shaped by the tragedy. Or strongly emphasizes its consequences.
Examples: Snape, Kakashi, Elsa (for a time), Scrooge 2017 (for a time), Amon, Jeong Jeong, Rust Cohle, Doctor Strange (sort of), Darth Vader, Goblin Slayer. Lelouch is a similar drama queen.
Walling Off
Another sub-theme is “deliberately building a wall between yourself and others”.
Some examples:
Severus Snape. A. Obsessed with Lily, closed to new romantic relationships. C. A loner in general. Partially due to his spy job. CA. Master at closing off his mind (Occlumency). E. His abrasiveness is meant to put off.
Kakashi from Naruto. C. Behaves reclusive. D. Dwells on his losses instead of trying to move on. E. Looks detached.
Elsa from Frozen. B and C. Her secret forces her to be distant from her sister and other people. E. Cold.
Lelouch from CodeGeass. B and C and D. His true intentions and personality are hidden even from loved ones. Keeps his burdens to himself.
Amon from The Legend of Korra. Cut off his family, buried his emotions (towards his brother), hid his past and abilities.
Dramatism / Sentimentality
Another sub-theme is “dramatism” (excess, grandiosity, making everything more serious and emotional) and “sentimentality” (nostalgia about the past, holding on to personal connections, being dramatic in a vulnerable way, awe about the world or history). Dramatism without sentimentality would be Thanos.
Severus Snape. Dramatic love, excessive grudges, dramatic role (playing a villain till death).
Lelouch from CodeGeass. Childhood tragedy, dramatic style (e.g. playing the villain and punishing himself), dramatic thoughts/feelings, grandiose goals.
Scrooge McDuck. Excessive wealth, legendary adventures, radical attempts to find Della.
Amon from Legend of Korra. Personal drama, dramatic death, grandiose goal, showy methods.
Nicholas Van Orton from The Game 1997. Childhood tragedy, excessive wealth, insane “game”.
Elsa, Darth Vader, Gandalf, Goblin Slayer, Cigarette Smoking Man are other drama queens.
A music band example:
I Like Trains
On average, I Like Trains songs contain fragility, dramatism and sentimentality.
A Rook House for Bobby. Genius. Personal, social and psychological fragility. Dramatism (persecution, cold war) and sentimentality. “And if they find me / Let them indite me / I just don’t care any more (...) They’ve pushed me too far, too far (...) All this talk of war / But it’s only a game (...) All I ever wanted to do is play chess with you”
Terra Nova. Genius. Personal, physical and social fragility. Dramatism and sentimentality. “How could I / Have led these men to their demise? / And they just follow / Exploration’s last great prize / It wasn’t mine (...) And more’s the shame / You will remember my name”
A Father’s Son. Genius. Dead dad speech. “Pry these words from my cold dead lips / Home is where the heart is (...) I will occupy the space between / What you say and what you mean / And you say / That the devil’s in the detail / And if that’s true / When he is through with me he will / Sure as hell be after you”
Beacons. Genius. SPOILERS! A societal collapse and murder. The lyrics don’t follow the story 1-1, but mention some societal and personal fallout.
A Divorce Before Marriage. Genius. Abstract song about a breakdown of a dream, very dramatic/sentimental. “The echo of success / In the beating of a heart / Will tear your life apart / And your forgotten dreams / As futile as they seem / Gave you hope / Now that hope is not enough / All the people that you meet / Are as hollow as they seem (...) Delusions of grandeur / A divorce before marriage / An open wound”
Death of an Idealist. Genius. Social and psychological breakdown. “This is a breakdown / You are a cancer / This will be the death of me / I’ll never live this down / You will never / An amazing fall from grace / No more or less than I deserved / A gentleman, a liar and a cheat”
The Voice of Reason. Genius. “It took a blow to the head / And I realized what I / Was put on this earth to achieve / It showed me the way / So make an example of me / Hang me up from the tallest tree / Make an example of me / This is not the voice of reason / It’s the cross I was chosen to bear”. About James Hadfield.
One theme (liminality) and 3 sub-themes (ego, entanglement, devaluation). 18+ examples.
Liminality
Theme of the next archetype is liminality (transformation, transition, detachment, isolation, being stuck). A perfect example of this theme would have the following properties:
A. Being stuck in a place or some neverending process. Living in a transforming world.
B. Personal relationship going through or causing a major transformation.
C Going through the transformation of social status. Being alien to people around.
CA. Physical transformation.
D. Psychological transformation. Being mentally stuck in a weird/uncertain place.
E. An attitude which mixes strong emotions and absence of emotions.
In other words, liminal life, liminal relationships, liminal social and “physical” status, liminal mentality, liminal attitude.
Examples:
Zuko from ATLA. A. Stuck in a neverending goose chase. B. His relationship with Aang goes from rivalry to friendship. C. A banished prince hoping to restore his honour. D. Becomes really conflicted about Good and Evil and his Purpose. Changes alignment. E. Cocky and impatient, but stoic. Zuko’s Complete Timeline (up to 11:00).
CC from Code Geass. A. Stuck in this world, unable to die. Stuck seeking someone capable and willing to kill her. B. The development of her relationship with Lelouch is about finding the will to live. C. An immortal witch stuck in a world she doesn’t belong to, getting tortured and experimented on. Helps the revolution she doesn’t care about. D. Her wish is to die. Her existence is just lingering. E. Both brazen and unemotional. You can watch 4 minutes (5:26 − 11:17) to get her attitude.
Gaara from Naruto. B. His relationship with Naruto is about him changing moral alignment. C. He’s used as a tool in a war he doesn’t care about. CA. He’s always on the verge of losing control and transforming into Shukaku. D and E. Gaara is in a severe distress, due to childhood traumas and internal fight with Shukaku. He has no attachment to anything in this world, his only goal is “to kill to assert his own existence”. Cocky and unemotional, calm and unstable. Sasuke vs Gaara, 7 minutes.
Lee Woo-jin from Oldboy. SPOILERS. B. His “relationship” with Daesu is about transforming Daesu’s life. CA. Contrasts with all other characters (criminals and poors). D. Lives only to wait for revenge and death. E. Cocky and unemotional. Ending scene (4 minutes), more ending (3 minutes).
Alex from Clockwork Orange. A. Most of the movie he’s stuck in a weird journey without a home. B. His relationship with droogs and parents transforms radically. C. Gets jailed, turns into a subject of experimentation, loses his home. His relationship with society transforms 180 degrees. D. Most of the movie he spends in a weird reprogrammed state, unable to do evil, but not truly good. E. Both cocky and unemotional. For example, whistands abuse calmly. Mr. Deltoid scene, 2 minutes.
Draco from HP. B. Harry turns from Draco’s rival to one of the only people who can save him and his family. C. Stuck between the good and the bad side. Doesn’t belong with the bad guys, they’re too extreme for him. His family got sidelined, lost their status among the Death Eaters. Also, in most of the books he’s somewhere on the sidelines. D. Very distressed and confused in the later books. E. Cocky and cold.
Harry Osborn from The Amazing Spider-Man 2. Loses status (Oscorp company), health (genetic disease), sanity (goes Green Goblin), morality. Daddy issues. Coldish and prideful. Meets dying father (4 minutes), asks Peter (3 minutes), meets Spiderman (2 minutes).
Dorian Grey from the 2009 movie. B. His relationship with Henry is about his moral deterioration. His relationship with Emily is about his redemption. C and D. Stuck in the immortal hedonistic lifestyle, without any purpose. Conflicted between Good and Evil, changes alignment multiple times. E. Cocky and unemotional.
Once-ler from The Lorax 2012. A. Symbolizes the growing danger of capitalistic greed. B. His status in the family radically transforms. C. His social status radically transforms. D. His morality radically transforms. How bad can I be? E. His attitude radically transforms. The biggest Tumblr Sexymen (other than Sans). A video explaining Onceler and Oncest in 2 minutes.
Lord Shen from Kung Fu Panda 2. A. Brings the new era, the era of gunpowder weapons. B. KFP 2 is sort of about Po’s relationship with him transforming. From PTSD to “inner peace”. The same can’t be said about other KFP villains, He has the most personal connection to Po. C. He’s a banished prince, but returns his status by taking over his home town. D. Desperate to satisfy his incomplete ego. “The cup you choose to fill has no bottom. It is time to stop this madness.” (c.) Soothsayer. E. Cold and elegant, but short-tempered and cocky. Shen vs. kung fu masters (2 minutes), Shen’s fortune (3 minutes).
Lapis from Steven Universe. A. Was stuck in a mirror for thousands of years. Then was stuck on the bottom of the ocean, fused with another gem (Jasper). Later she’s free, but has no home. At some moment she hides on the Moon. B. Takes a long time to befriend crystal gems. C and D. Was aligned with an evil empire (Homeworld), killed sentient life by terraforming planets, but wasn’t truly evil or a pro-empire diehard. Wanted to protect the human (Steven) who freed her. She was sorta like a civilian scientist in Nazi Germany. Eventually she changes her alignment entirely. Has tons of trauma and depression. E. Very calm and very angry/disheveled. Lapis clips.
Dio from JoJo Phantom Blood. B. His petty rivalry with Jonathan transforms into a global fight between Good and Evil. C. He’s an adopted son in a family he doesn’t care about. CA. Transforms into a supernatural evil being. E. Cocky and calm. 4 minute clip.
Seita from Grave of the Fireflies. A and C. Seita and Setsuko lose their home and wander without a (true) home. Their social status gets worse and worse. First they live with an abusive aunt, then they’re on their own in an abandoned duguot. B. His relationship with Setsuko transforms, he becomes her guardian and the only friend. E. Appears unemotional, but a lot of bottled feelings.
Kiki from Kiki’s Delivery Service. A. The movie is about growing up and depression / artist’s block. B. Has uncertain friendship with Tombo, connected to her loneliness/depression. C. Kiki gets into a new town. First time on her own, first job. Being a special “race” (witch) and having more reponsibilities (a job) contributes to isolation. D. Lonely. Later depressed. E. Calm but prideful.
Kurisu from Steins;Gate. B and D. Goes from antogonizing to loving Okabe. C. The most prestigious and mature in the group. The whole show is her being taken out of her comfort zone, dipped into mad science & conspiracy & weirdos. E. Unemotional but prideful.
Sheldon from The Big Bang Theory. B and C. Detached in social and personal life. Has the constant dynamic where he keeps getting dragged into uncomfortable situations and HE keeps dragging others into uncomfortable situations. E. Unemotional/shy but aggressively prideful. Sheldon Cooper Moments from Seasons 1-6.
Luka from Miraculous. B. Has liminal love/friendship with Marinette. Supports her love for Adrien but still has feelings for her. B. Has split family. C. IIRC somewhat disconnected from Marinette’s main friend group. Learns Marinette’s/Adrian’s secret and leaves Paris to protect it. CA. Gets the superhero power to indefinitely rewind time to a checkpoint. D. Has unusually strong empathy which makes him a bit isolated as a mediator/therapist. But who’s gonna help him? E. Appears way more untroubled than he truly is. See 4:28 − 9:42.
Randall from Mosnters Inc. A. Works on a conspiracy to change everything. “I’m about to revolutionize the scaring industry. And when I do, even the ‘great’ James P. Sullivan’s gonna be working for me.” (c.) B and C. Constantly 2nd place to Sally. CA. Can turn invisible. E. Can be calm/sly or explosively spiteful.
Real-life semi-example:
Ding Liren. A. Became the world champion in a liminal period, where Magnus exited undefeated. C. Abruptly lost strength. D. Had mental health struggles. E. Appears unemotional.
Semi-examples:
Eric Draven from the Crow 1994. C. Undead. E. Both unemotional and explosive. Eric from the 2024 movie would be a proper example.
Fubuki from OPM. Overshadowed by her sister, prideful and cold.
Lord Beckett from POTC. A and CA. Symbolizes the transformation of the world, the dawn of the pirate era, magical era. “This is no longer your world, Jones. The immaterial has become… immaterial.” (c.) He, a businessman, doesn’t belong in the world of pirates and magic. His motivation is a bit alien compared to other characters. E. Both cocky and unemotional.
Westley from The Princess Bride. A. Transformed from a regular farm boy into one of the most dangerous men in the world. The entirety of the movie he’s stuck in a neverending rescue mission. B. His relationship with Buttercap transforms from “master and servant” to “captor and captive” to “superhero and damsel in distress”. His relationship with Inigo and Fezzik transforms from enemies to friends. C and CA. He’s pretty alien, so unusual physically and mentally. A big chunk of the movie he hides his identity, looking like some weird ninja. Another chunk he’s recovering from death. E. Both cocky and unemotional. This 7 minute scene is an example again.
Silas from The Da Vinci Code. “I am a ghost.” A and C. His whole life he’s a hidden outcast. CA. An albino. D. Stuck in the state of violating his principles and punishing himself. E. Hot and cold.
Rei from Evangelion. TV series SPOILERS. She’s a clone and a tool. Unemotional.
Squidward from Spongebob. C and D. Longs for celebrity life, but it never comes. Uses pessimism and misanthropy to cope. E. Cold and cocky.
See below. Eric doesn’t have enough dimensions of liminality, as well as not enough ego and devaluation. Lord Beckett, Westley—not enough devaluation. Rei—not enough ego. Silas, Fubuki—not enough ego and devaluation. Squidward—surprisingly, not enough ego… he is ready to compromise with his ego way too much and is not proactive enough.
Ego
Liminal!isolation can be caused by ego (pride, privilege, elitism, superiority).
Zuko from ATLA. B. Looks down on Iroh, because Iroh doesn’t display ambition. C and D. Motivated by restoring his royal privilege and honor. Fire Nation suprematist, calls Katara a peasant. E. Cocky.
CC from Code Geass. B. Treats Lelouch as a source of amusement. C. Has some privillege due to immortality, tho it’s questionable. E. Behaves like a queen.
Draco from HP. B. Jelous of Harry. C. Spoiled brat. D. Blood supremacist. E. Cocky.
Dorian Grey from the 2009 movie. Vain hedonist narcissist, in personal relationships and in general.
Dio from JoJo Phantom Blood. Cocky. Feels entitled to everything Jonathan has. Even steals his girlfriend’s first kiss. Kono Dio Da.
Lee Woo-jin, Lord Shen (KFP 2), Harry Osborn, maybe Seita (Grave) are other strong examples.
Entanglement
Another sub-theme is “entanglement”. When your fate is unusually entangled with another person.
Some examples:
Zuko from ATLA. His social status and family matters and morality all get entangled in his rivalry with Aang. Zuko even sees himself as Aang in one dream.
CC from Code Geass. Her fate, status and goals/motivations all get entangled with Lelouch.
Lee Woo-jin from Oldboy. His personal life, criminal activities, future plans/ambitions… everything is about Daesu.
Draco from HP. His status, morality, fate of his parents… all become dependent on his rival, Harry.
Lord Shen from Kung Fu Panda 2. His fate, status, ambitions… all become dependent on Po.
Even lovers don’t often have this much entanglement.
Harry Osborn and Peter Parker, Dio and Jojo, Seita and Setsuko, Kurisu and Okabe, Randall and Sally are other strong examples.
Devaluation
Liminal!pain is about absence of expected value. Deprivedness, disrespect, depression, marginalization, dissatisfaction, etc.
Zuko from ATLA. B. Iroh is Zuko’s closest connection after banishment. Zuko cared about him, but still saw little in him. “I think you are exactly what you seem! A lazy, mistrustful, shallow old man who’s always been jealous of his brother!” (c.) C. He’s a prince, but not respected. D. His purpose (regaining honor and father’s respect) starts to become meaningless to him. E. Shows no respect to superiors (e.g. Zhao).
CC from Code Geass. B. Helps Lelouch, but doesn’t care about his plans. D. Doesn’t see value in living anymore. E. Unusually unbothered by the stuff happening around her.
Gaara from Naruto. A. Was a nice kid, but snapped after he was convinced he wasn’t ever loved by anyone. B. Didn’t care about his siblings. They were terrified of him. C. Was a big element of an invasion plot he didn’t care about. D. He is pretty purposeless, his only purpose is to kill people “to prove his own existence”. Believes he never been loved (the symbol on his forehead is “love”). E. Unusually unbothered by attacks (due to the automatic sand defense).
Alex from Clockwork Orange. B. Becomes inconsequential to his gang and family. C. A part of society, has parents, goes to school and clubs, but doesn’t care about people (abuses and kills them). D. Loses his purpose. E. Shows no respect to anyone.
Lord Shen from Kung Fu Panda 2. B. Thought his parents hated him. C. Was a banished prince. D. Has insatiable ego. “The cup you choose to fill has no bottom. It is time to stop this madness.” (c.) Soothsayar. E. Peacocky.
Lapis, Dio (JoJo), Harry Osborn, Seita (Grave of Fireflies) are other strong examples.
A very similar sub-theme is “loss of agency” (getting stuck, losing resources / support, getting depressed, etc).
One theme (punkness) and 3 sub-themes (social conflict, uncomplicatedness, direct attachment). Because of the sub-themes “punkness” turns into something more like “soft punkness”. 15+ examples.
Punkness
Theme of the next archetype is punkness. More specifically, “subverting social properties of status”. High social status is desirable; social status is relatively rigid; high status people hook up with other high status people; high status people are powerful and talented, etc. This archetypes subverts all that. A perfect example of punkness would have the following properties:
A. Radical changes of social status.
B. A relationship against social rules.
C. Fighting against high-status people or organizations or the order of society.
CA. Charisma and great skills despite low status. Chaotic fighting style or self-expression. Status-subverting abilities.
D. Simple goals and approach. Dislike of high status.
E. Punk attitude, low-status attitude.
In other words, punk life, punk relationships, punk social role, punk mentality, punk attitude.
Examples:
Sirius Black. A. Run away from his wealthy racist family. C In Book 3 he was alone against dementors, authorities and Hogwarts defenses. CA. Canonically charismatic. Can turn into a dog. D. Simple goal (help Harry / kill Peter) and straightforward approach. E. Has punk attitude (motorcycle, posters of pinup muggle girls, pranks, taunting while dueling). A bit crazy and immature.
Robin Hood from the 1991 movie. A and C. Was a nobleman, became the king of thieves, robbing wealthy and giving to the poor. B. His love with Marian has status contrast. CA. Canonically charismatic. D. Simple goals and approach. E. Very bold and somewhat unserious.
Beetlejuice. B. Obsessed with a little girl. CA. Has a lot of power despite being a dead hobo. Acts wildly and whimsically, can shapeshift. D. Simple goals and approach. E. Funny, offensive, crazy. His introduction, 2 minutes (2:00 − 5:30).
Daesu from Oldboy HEAVY SPOILERS. A. Lost all of his social status by getting abducted, jailed and framed for murder. Was an alcoholic, turned into a martial arts master. B. His love with Mido violates social rules, in the worst possible way. His “relationship” with the antagonist has status contrast. C. Fights, all alone, against organized crime and a wealthy adversary. CA. Compensates lack of skill/technique with rage. D. Simple goals and approach. E. Looks and behaves wild.
Alucard from Hellsing Ultimate. A and C. Was a sex slave, became a christian warlord, but betrayed his humanity in a moment of weakness and turned into an immortal overpowered vampire. Then was tamed and started working on a christian organization, fighting other evil beings. Wild status changes. B. Despite being infinitely more powerful than Integra, he’s her servant. CA. Sadomasochistic fighting style. Allows opponents to mutilate him and humiliates them before killing. Can shapeshift. D. Wants to find a worthy opponent to die to. He’s bored of immortality and considers himself a failure (for becoming a monster). E. He’s a troll.
Joker from The Dark Knight. A. Was a nobody, became the main criminal in town. B. His weird “relationship” with Batman has status contrast. C. Tries to ruin the social order. CA. Unpredictable actions, “improvisation” (even though it’s fake), disguises. D. Simple goal (chaos). E. Looks and behaves wild.
Sherlock Holmes, Guy Ritchie version. B. Loves a criminal (Irene Adler). C. A mix of a detective and a vigilante. Too much collateral damage. E. Punk attitude. Opening scene (6 minutes), with Irene (3 minutes).
Wendy from Gravity Falls. B. Age gap friendship with something romantic (though she doesn’t reciprocate). C. Unusually tough kid (15 years). Lumberjack games, survival training, stressful home with no mother. Skilled in combat. Slacks on the job, breaks the rules, breaks the law (a little bit). E. Punk attitude. Her clips, 2 minutes.
Marceline from Adventure Time. A. Rejected her dad (low-key Devil) and reign over the Nightosphere (basically Hell). B. Lives in a simple house in a cave. Her girlfriend has a castle and a kingdom. CA. She’s a punk rocker. Can turn into a monster bat. D. No complex goals. E. Punk rocker, trickster, down to earth. “I’m not mean. I’m a thousand years old, and I just lost track of my moral code.” Evicted (1:00 − 3:40), The Henchmen (5 minutes).
Inigo Montoya from The Princess Bride. C A reluctant criminal with a righteous goal and exceptional swordfighting skills. CA. Charismatic, quickly wins Weastley’s respect. Likes showing off and taunting while fighting. D. Simple goal and approach. E. Somewhat unserious, impatient, cocky and daring. 7 minute scene.
Speedwagon from JoJo. A. Was a street thug who became a wealthy businessman. B. Became an extremely loyal friend to a rich family. C. Very honorable despite being a thug. D. Straightforward mentality. E. Has the attitude. His first scene (3 minutes), exposing Dio (2 minutes).
Korben from The Fifth Element. A and C. A former special forces major becomes a taxi driver, then joins the mission to save Earth. Fights against an army of a corporation (Zorg and Mangalores). B. Manages to woo a perfect being. D. Straightforward mentality. E. Has the attitude.
Bumi Jr. from LOK. C. Has status (commander), yet he’s a comic relief. CA. Unorthodox tactics. D. A bit submissive, has self-worth issues about lacking bending powers. “Look, uh, I’m sorry I didn’t turn out to be an airbender like you hoped. But I’ve tried my best to keep the world safe. Hope I made you proud.” E. Wild, comical. Saves everyone (5 minutes), talks to Aang (30 seconds)
Eric from The Eric Andre Show. C Takes a high-status concept (late night shows) and turns it into a shitshow. CA. Chaotic. D. Humiliates others and himself. Straightforward approach. E. Punk. 3 minute example, another 3 minutes.
Real-life example:
Mikhail Tal. Incorrect and whimsical playing style, humor, disregard for health. But a champion-level strength.
Adam Sandler. C and D. Talented actor, but ready to waste his talent on shitty movies. Doesn’t treat movies as seriously as other actors. E. Has punk attitude and low-status appearance (popularized “grocery-run look”). On Oscars 2025. About his movies.
Michael Jackson. Semi-example. B. Creepy relationships with kids. C. Was a profilic philanthrope and promoted social causes in songs too (e.g. Man in the Mirror, They Don’t Care About Us). CA. Popularized fancy moves (the moonwalk, the robot, the anti-gravity lean) and music videos. D. Mentally stuck in the childhood. E. Charismatic but weird, down-to-earth.
Semi-examples:
Charlie from All Dogs Go to Heaven. A. Escapes prison, loses money, escapes heaven, becomes rich again. C. Low-life. D. Straightforward mentality. E. Punk attitude.
Fenton/Gizmoduck from DuckTales 2017. B. Loves a criminal. C. Starts as an unpaid intern with a lab in the bathroom, becomes a respected hero. D and E. Straightforward mentality, naive and earnest. Awkward, clumsy attitude. Fenton clips.
Fujimoto from Ponyo. A. Rejected humanity and became the sea king. B. A human who rizzed a goddess. C Despite being a king he’s sorta a comic relief. His kid dosabeys him and he can’t do much about it. Submissive to his wife. CA. Looks unkempt and acts whimsically. Not afraid to show up on the surface and do the dirty work himself.
Billy Madison 1995. A and C. A wealthy father with a big company has a poor-looking buffoon son, Billy, who has a chance to inherit everything. Billy goes back to elementary school, as an adult, to prove himself. B. In love with his elementary teacher. D. Doesn’t care about living seriously. Rejects the inheritance. Despite being an idiot he has a good heart and hidden smarts. E. Low-status behavior.
Han Solo from SW. A and C. A smuggler who becomes a rebel general. B. Woos a princess. E. Punk attitude.
Tom Bombadil from LOTR.
Julian Rush from The Penguin. B. Woos a powerful girl. C. Psychiatrist turned mafia goon. D. Unexpectedly submissive.
See below. Fenton, Billy, Tom B, Rush—not enough deliberate conflict. Charlie—maybe not enough conflict too. Han, Fujimoto—not uncomplicated enough, too much responsibility and grumpiness. Michael Jackson—not enough conflict.
Social Conflict
Punkness!isolation is caused by “deliberate conflict”.
Examples:
Sirius Black. A. Run away from his wealthy racist family. B. Bullied Snape, mistreated Kreacher. C. Troublemaker, bully, assumed criminal. D. Revenge-driven. E. Somewhat arrogant (e.g. taunting while fighting).
Robin Hood from the 1991 movie. B. Fucks with Nottingham. C. Does crimes. D. Partially revenge-driven. “I’ll not rest until my father is avenged. I swear it by my own blood.” (c.) E. Brash.
Beetlejuice. B. Tries to wed Lydia against her will. C. His “job” is bio-exorcism. He scares and might kill. E. Brash, offensive.
Daesu from Oldboy. B. Trapped in an urgent conflict with Lee Woo-jin. C. Fights organized crime. D. Revenge-driven. E. Brash, offputting.
Inigo Montoya from The Princess Bride. C. A criminal. D. Revenge-driven. E. Impatient and cocky.
Alucard, TDK Joker, RDJ Sherlock (offensive, insufferable, kicking ass) are other strong examples.
Uncomplicatedness
Another sub-theme is “seeming effortless / lacking usual responsibilities / being submissive / straightforward / relaxed” (uncomplicatedness).
Examples:
Sirius Black from HP. B. Unusually devoted to Harry. Harry is canonically the person Sirius cares about the most. C. Waited 12 years in Azkaban, then was under home arrest. Sometimes freely roams as a dog. D. Straightforward mentality. E. Can be very soft.
Robin Hood from the 1991 movie. C. Lives in his utopia in the woods. CA. Most of the time dispatches enemies effortlessly. D. Straightforward mentality. E. Laidback, down-to-earth.
Beetlejuice. C. Has no real job or responsibilities. CA. Dispatches people effortlessly. D. Straightforward mentality. E. So relaxed he ignores personal boundaries.
Daesu from Oldboy. C. Was jailed for 15 years. D. Straightforward mentality. E. Can be submissive.
Alucard from Hellsing Ultimate. A. Was a sex slave. B and C. Submissive to Integra, serves her organization. CA. Dispatches people effortlessly. D and E. Straightforward mentality. Masochistic. Sees himself as inferior to dignified humans. Can dish out a good complement—“I still see the same spirited young woman I met fifty years ago, Your Majesty. In fact, in my eyes, time has made you even more beautiful, your highness.” (c.)
TDK Joker, RDJ Sherlock, Wendy, Marceline, Inigo Montoya, Speedwagon, ADGTH Charlie are other strong examples.
Direct Attachment
Another sub-theme is “direct attachment”. Unusually strong, direct, multi-faceted attachment.
Sirius. Protects Harry, cares about him the most, was ready to house him.
Robin Hood. Protects Marian physically, socially (removes Nottingham), knows her from childhood, swore to protect her to her dying brother.
Daesu from Oldboy HEAVY SPOILERS. Loves Mido, protects her physically, etc.
Alucard from Hellsing Ultimate. Serves Integra, her family and organization. Protects her. Admires her.
Speedwagon from JoJo. Helps Jojo, accompanies him, admires him, leaves criminal life behind for him. Later helps his family.
Korben (loves / protects / helps Leeloo), Inigo Montoya (devotes his entire life to avenging his father), ADGTH Charlie (protects Anne-Marie, finds her a family, visits her from Heaven) are other strong examples. Beetlejuice (obsessed with Lydia), Joker (obsessed with Batman) are weaker examples.
If you’re interested, you can message me on LW or Discord.
The post inspired me to think about the differences between abstraction types. This is relevant to my own research too, so I got stuck on (re)writing this comment for more than a week. The main idea I converged to: we should analyze abstractions through things which make them useful. The first section below is the most relevant to the OP, it contains 4 potential distinctions between physical and mathematical abstractions.
Distinction 1A. A physical abstraction predicts only a specific aspect of a situation or a trivial sum of specific aspects. A mathematical abstraction is useful for proving non-trivial facts about entire situations. Basically the same abstraction can be both physical or mathematical, depending on how it’s used. Examples:
“Ants” is a physical abstraction. “Moving points” is a mathematical abstraction when used to prove a theorem about ants on a stick. After you’ve proved the theorem, you can easily predict how an infinite number of situations would unfold.
“Number of sheep” is a physical abstraction when used to track the number of sheep.[1] “Numbers” is a mathematical abstraction when used to show that you never know when you can split your sheep into equal groups with more than 1 sheep in each (because some numbers are prime and there’s an infinity of primes).
“Forces” is a physical abstraction when used to predict the movement of a pendulum. “Vectors” is a mathematical abstraction when used to prove the hairy ball theorem.
“Programs” is a physical abstraction when used to predict physical computers. “Programs” is a mathematical abstraction when used to prove the insolubility of the halting problem.
“King safety” is a physical abstraction when used to predict one aspect of the chess position (whether the king is likely to get checkmated soon). “King safety” is not a mathematical abstraction when used as a part of an evaluation function, predicting the development of the entire game. Because you can’t use king safety to prove what the evaluation function should be.
“Elastic ball” is a physical abstraction. “Elastic sphere” is a mathematical abstraction when used to prove the Borsuk–Ulam theorem.
Distinction 1B. Physical abstractions refer to groups of objects existing on roughly the same scale and behaving similarly, groups conceptualizable as “singular entities”. Physical abstractions capture properties of those objects which are directly valuable/impactful. Mathematical abstractions do this much more rarely. Their main purpose is to reveal or transfer non-trivial facts about objects.
Referents of “animals”, “minerals”, “cars” exist on roughly the same scale and behave similarly. “Animalness” of an object makes it valuable and impactful. “Animals” can be thought of as a sort of singular entity. Referents of “vectors”, “functions”, “metric spaces” exist all over the place. “Vectorness” of an object doesn’t make it valuable or impactful. Both gravity and noses can be “vectors”, but you can’t think of “gravity + noses” as a coherent entity. Mathematical abstractions are too abstract, they bundle too much dissimilar stuff.[2]
This is a reframing of your Take 4 and suffers from the same problem (e.g. “animals” also exist in books and “real animals + book animals” is not a coherent entity). Though maybe it makes a stronger case that the problem is merely an edge case.[3]
Distinction 1C. Physical abstractions are inside the same causal system, they shape each other by adding causally linked details to each other. Physical abstractions are implicitly reflected in each other, like jigsaw puzzle pieces or the Indra’s net. For example, trees shape animals and animals shape trees. Mathematical abstractions can be combined into a single system too, but they won’t have the same interconnectedness. (This is a variation on Takes 1 & 3 and feels related to the well-abstractibility prior.)
From a drop of water a logician could infer the possibility of an Atlantic or a Niagara without having seen or heard of one or the other. (c.) A Study in Scarlet[4]
Distinction 1D. Physical abstractions wouldn’t be useful if there were too many of them and they were too difficult to recognize. For example, if you wanted to get some berries in a forest, but there were too many types (with wildly different properties) which were too hard to recognize, then knowing the types would be practically useless. Therefore, there should be something limiting the diversity and hardness-of-recognition of important physical abstractions. Meanwhile, in math you can create infinities of unrecognizable objects (e.g. uncomputable numbers and unpredictable programs). 1D is related to 1B and 1C.
Adjectives wouldn’t be useful if the contents of your visual field (and other perceptual fields) didn’t tend to follow certain “composition rules”, such as “different color patterns tend to correspond to different physical abstractions”. Moreover, it’s probable that composition rules themselves wouldn’t be useful if we didn’t have “meta composition rules” to decide which composition rules to apply. This is related to truesight. Some of my previous comments on the topic: one, two. “Adjectives” and “(meta) composition rules” are different types of abstractions, the same way “mathematical abstractions” and “proofs & situations-where-proofs-are-applicable” are different. Technically independent, but the former wouldn’t be useful without the latter.
Distinction/definition 2A. An adjective doesn’t reveal non-trivial facts and is far less predictive than a physical (noun) abstraction. An adjective doesn’t reference a coherent group of objects (e.g. consider the set of all “red” objects or the set of all “big” objects). A composition rule reveals non-trivial statistical facts and references a group of arbitrary situations (which can be split into subgroups of arbitrary objects close in spacetime). Ditto a meta composition rule.
I guess we can keep defining abstraction types. For example, why are verbs useful? Partially for the same reason adjectives are useful (certain actions (transformations of the world) make physical abstractions easier to recognize). And partially because certain actions are important regardless of what does them. For example, if something is moving towards you, it’s very important regardless of whether it’s an animal, a rock, liquid or gas.
A big inconvenience is that the definition of an abstraction type might depend on arbitrary other abstractions. Maybe the most complex philosophical reasoning is about discovering new abstraction types, customized for a specific problem you’re solving / a specific level of reality you’re analyzing.
If that’s correct and we can’t directly classify all abstraction types, what can we do? I hope we can study how mismatch between abstraction types leads to misalignment. Studying what makes abstractions entangled with our universe (OP’s theme) is a special case of that. In the pessimistic case, even the special case would require finding a set of abstraction types which can express any abstraction type (akin to Turing completeness).[5]
In the last section I’ll try to explain the most ambitious hope about all this. Those are the most unfinished thoughts.
One topic I’m interested in is something like “emergence (of meaning) in abstractions”. When an abstraction feels qualitatively different from the things it’s abstracting over or the things it’s composed of. Or when an instance of a general abstraction feels qualitatively different from all other instances. Put differently, sometimes it feels like the definition of an abstraction feels much more important than the aggregation of things fitting the definition—as if we’re making a creative leap from examples to the general theme. This might be related to semantic frames and to holism vs. molecularism in general (“can a word be defined independently from the entirety of the language?”).
I can’t give a good example, so here’s a bunch of average ones: “aggression” (more than “dangerous directed actions”), “happiness” (more than a type of pleasure), “kindness” (more than a type of helpfulness), “corrigibility” (ditto), “shit” (more than unpleasant/useless matter), “running” (more than a “fast sequence of actions in a direction”), “murder” (more than a type of destruction), “general intelligence” or “sapience” (more than a sum of specific skills and feelings), “scary” (doesn’t feel like a generalization of particular scary things or scary features), “kafkaesque” or “inconvenient” in general, “cheesy” or “cheap” in general, “harm” or “trauma” (more than a type of lasting disutility), “fragility” (more than a type of instability), “irony” (more than a type of contradiction), “massive” (more than “very big”). Feels like lots of words in natural language have an extra informal oomph. Metaphors (“love is war”, “job is a jail”, “price is rising”) might be one of the main drives behind it. Even if all this is unnatural, there’s gotta be some general psychological mechanism by which concepts get wrapped in extra layers of meaning which feel so intuitive, right?
Maybe my main idea (“analyze abstractions through things which make them useful”) could help to analyze this. For example, “massive” means more than “very big” because “massive” is useful in more ways than a generic adjective. It helps to quickly recognize objects (like generic adjectives do) but also helps to quickly predict important outcomes (like generic verbs do). If you meet a massive bear, you’re unlikely to beat it. If you encounter a massive mountain, you’re unlikely to cross it. If you have a massive debt, you’re unlikely to pay it off.
If we’re lucky, this type of analysis generalizes really well and we can interpret (almost) any thought like this. If we’re even more lucky, this type of analysis naturally leads to a more specific version of ARC-style correlation analysis (it should be plausible because types of abstractions correspond to types of correlations).
If you have N sheep, you can make a prediction that you’ll most likely have N sheep at a later time. But it’s a trivial combination of each sheep’s object permanence.
Mathematical abstractions are made useful by proofs and situations-where-proofs-are-applicable. Proofs don’t refer to any physical objects at all. Situations-where-proofs-are-applicable, like mathematical abstractions themselves, refer to very incoherent groups of physical objects.
Also, maybe “computations about animals + computations about book animals” is a much more coherent entity. But then we need to define the notion of “scale” for computations.
The quote doesn’t actually help to differentiate the physical and mathematical abstractions, but Holmesian deduction in general is relevant to 1C.
Like what you already considered here:
Suppose that this symbolic-to-us language would be suboptimal for compactly representing the universe. The compression process would want to use some other, more “natural” language. It would spend some bits of complexity defining it, then write the world-model in it. That language may turn out to be as alien to us as the encodings NNs use. The cheapest way to define that natural language, however, would be via the definitions that are the simplest in terms of the symbolic-to-us language used by our complexity-estimator.
I believe you should be able to make progress by writing out different map-level and territory-level things. Let me try:
0) My prediction: I expect the universe to continue being ordered.
1) In-map explanation for why I came to believe it: I use a simplicity prior to predict my observations and trust my memory.
2) In-territory explanation for why my prediction succeeds: I’m a human (not a BB) in a simple universe.
The problem: I can’t trust my memory and justify the simplicity prior.
3) In-map justification for using the simplicity prior anyway: memory and simplicity aren’t provably reliable, but I don’t care about my life in case they are not reliable.
Potential crux A: (3) doesn’t undermine the fact that I’m correct only because of (2), so changing what I care about doesn’t change the outcome, yet it still counts as a justification of doing (1). If I cared about complex universes, I wouldn’t be justified in doing (1).
4) In-territory justification for using the simplicity prior anyway: I live in a multiverse where memory and simplicity aren’t reliable in every universe, but I don’t care about universes where they are not reliable.
5) Another in-territory justification for using the simplicity prior: God or some power makes the simplicity prior true in the multiverse.
Potential crux B: some will say (4) is not an explanation because I have to explain “how you go from experience 1 in universe 1 to experience 2 in universe 1 instead of experience 2 in universe 2” or something; others will say it’s all anthropic woo and any reasonable (5)-type theory is gonna be untestable by construction (until we die).
6) A deeper in-territory explanations for why my prediction (0) succeeds: “God/power did it” (see 5), “I got lucky once to end up in an ordered universe” (see 4), “all experience histories created by the multiverse exist, therefore this experience history was fated to be experienced anyway” (see 4). Due to crux B, I assume the latter can be disputed. The middle one can be criticized for being too lucky.
Why isn’t 3⁄4 a justification? Because of some philosophy regarding the flow of observer moments (crux B)?
He has a temptation to believe in simulation. But doesn’t he also have a counter-temptation to believe he’s the real Elon, not a random dude?
Not all epistemological things can be justified (rejecting modest epistemology, simulation hypothesis, Pascal mugging, other weird anthropic scenairos). At least it’s very unclear if they can be. So why should this one be justified? What kind of “Hilbert program” are you pursuing, how much do you want to justify in which foundation?
How likely is it, actually, that you are the real Elon Musk, vs. just some dude
Wait, isn’t this a bit of a too large humble pill to swallow?
“I just look at them with my eyes and remember who they are.” Should the alien conclude that the human is being uncooperative, and apply the crowbar?
With the crowbar on the table I’ll at least explain that the face splits into other high-level features (eyes, nose, mouth, etc.) which can be very memorable on their own.
As the bar inches closer I’ll venture into something more vague, about contrasts of features or associations or how features affect emotions.
...But that misses the point. On the topic of the tweet: doesn’t Eliezer acknowledge what you talk about later?
But, sure, maybe all the things that LLMs know about gravity and medicine and cups on tables is one kind of knowledge to them, and all the wordless secrets they know about humans are in a hidden and incommensurable format.
Eliciting Latent Knowledge problem for the unfamiliar:
Suppose we train a model to predict what the future will look like according to cameras and other sensors. We then use planning algorithms to find a sequence of actions that lead to predicted futures that look good to us.
But some action sequences could tamper with the cameras so they show happy humans regardless of what’s really happening. More generally, some futures look great on camera but are actually catastrophically bad.
In these cases, the prediction model “knows” facts (like “the camera was tampered with”) that are not visible on camera but would change our evaluation of the predicted future if we learned them. How can we train this model to report its latent knowledge of off-screen events?
So we have humans with their human ontology, the Predictor (the model we need to interpret), the Planner (can be ignored for simplicity), and maybe we’re training a Reporter (intended to interpret the Predictor).
We assume the Predictor is inner-aligned to myopically making predictions, it won’t strategically mislead us by outputting wrong information at the right time. We also assume that something strongly correlated with the diamond (Z) is important for the Predictor’s ability to make predictions (this is ARC’s working definition of knowledge).
I was wondering whether the problem is possible to solve with brute force, at least if we have unlimited compute:
Formalize the human model of physics. Let’s assume the model is true enough or we can check if it’s true enough.
Prove that the Predictor’s model reduces to the human model under normal conditions. This will give you a mapping between human ontology and the Predictor’s ontology.
We do something like this with models of physics. For example, we prove that general relativity reduces to Newtonian mechanics under normal conditions. With enough compute I could take String Theory and check all the 10^500 possible universes to find ours. Since the Predictor is inner-aligned, it can be treated basically as a model of physics… (= this might require making an additional assumption, but ARC is ready to make it I think)
I assume there are fundamental obstacles to this, but what exactly are those obstacles? Maybe we can’t define a universal language which would allow to prove the reduction of one model to another? Something else?
The most similar proposal to this I remember is “have AI help humans improve our understanding”. It is dismissed for being uncompetitive. Yet uncompetitive ELK is arguably not actually solved and there are deeper reasons to reject the proposal. It relies on more roundabout methods than just brute forcing how one thing maps to another.
Now, maybe I’m just restating the problem… but right now I don’t think so. I’m saying there should be a principled way to just prove how exactly one model reduces to the other instead of using arbitrary tricks to create a Reporter. If there are multiple, equally good ways to map human physics to the Predictor, then it would probably mean human physics aren’t grounded in anything at all.
...
Maybe one of the core problems is that the Predictor might be a mess of high-level and low-level variables? So we’d first need to find a way to transform the Predictor into a low-level model of physics / establish the hierarchy of variables. How hard is this problem?
I’d say “we don’t understand how the brain works good enough to replicate it in AGI” (and that means we lack a really important layer of understanding). Anyway, I think we agree here.
The point I wanted to make:
I think John’s argument for why corrigibility is natural works even if you don’t assume crisp separate modules.
Whatever explanation you propose (of the brain staying coherent), be it “your modules are corrigible to each other” or “the goals are kept at the back of your mind”, if you try to make that explanation more technical you’ll probably end up with a general solution to corrigibility.
Sorry for skipping so much inferential steps and causing confusion.
Sorry, why is this offtopic—what are the different problems I could’ve confused? The thing I quoted feels like it assumes exactly the reward hacking problem being solved:
the AI keeps the supervisor’s desires / preferences / aversions at the back of its mind, and notices when it has an idea that would bother or offend the supervisor, and then not do it
It assumes we have a thing which can accurately judge ideas as “good” or “bad”. I don’t believe it’s a solved problem. I went on to re-read the “act-based approval-directed agents” post, but it offered no solution. This reply to John seems to draw a distinction between good foresight and good hindsight, but I’d say both are unsolved problems and might have a similar solution. The latter is the credit assignment problem and I remember your post about it, with no general solution.
I’ll share my general view, maybe it’ll help to sort out the misunderstanding:
I think the brain is a horrible mess. “Why does the whole system stay relatively stable, without drifting uncontrollably into arbitrary directions?” is the core alignment-related mystery about it. “How is the brain able to correctly introspect its thoughts?”, “how is the brain able to notice failures/successes of its high-level goals?”, etc. are all facets of the core mystery. I believe they are all unsolved and not obvious. IMO any solution to any of the facets will give a (non-trivial) insight about corrigibility.
Anyway, does this suggest a solution to AI corrigibility? Well, kinda. But it’s just the obvious one: the AI keeps the supervisor’s desires / preferences / aversions at the back of its mind, and notices when it has an idea that would bother or offend the supervisor, and then not do it.
I think it totally does suggest a solution, if you could reformulate the quote in a much more abstract and general way, along with explaining how/why the AI doesn’t fool the supervisor. I was skimming the intro to brain-like AGI safety and IIRC the problem “how/why doesn’t the thought generator fool the thought accessor?” wasn’t solved beyond “create as many thought accessors as possible and hope for the best”.
In general, I’m very interested in the intersection between brain-like ideas and agent foundations ideas.
This is objectively the best one.
Questions felt like a better way to get feedback on some things (exactly because they are rare). I miss forums having sub-forums for the same reason.
But you’re not going to find any good references, or come to any conclusions, if you stay at this stage.
I think he already came to some conclusions and you already gave some good references (which support some of the conclusions).
For example there are ways of designing hardware which is reliable on the assumption that at most N transistors are corrupt.
Do those methods have names or address problems which have names (like the Byzantine generals problem)?
I guess my main problem with the post is that too many points are implicit/underexplained/have no examples.
This is an interesting point, but it felt random in the post. Like you just drop this point, disconnected from the main criticism.
I’m not sure AF researchers do this. Though I understand that calling out specific people would be offensive. Just saying that it’s not an obvious claim.
I cited EY’s AF work, because the post is about AF. The Sequences are not really AF work. I’m not sure people do AF work in the style of the Sequences? Examples would be nice. Meanwhile, here’re other counterexamples: ARC (Eliciting Latent Knowledge), Paul Christiano, Vanessa Kosoy, Alex Flint (optimization, accumulation of knowledge), TurnTrout (reframing impact), John Wentworth, Thane Ruthenis, decision theory / Lobian obstacle / logical induction work...
Definitely you’re either too charitable to analytic philosophy or too uncharitable to AF research. Or have in mind some non-obvious distinction.
In my experience, lots of analytic philosophy starts with thought experiments and non-axiomatic theories, then discusses possible counter-arguments to them (still not establishing any axioms). Sometimes it does infer “axioms” in the context of a specific debate, but anyone is free to disregard them or question their implications (because those “axioms” are not really axioms in some logic). See
EP and NEP theories
meaning holism vs. molecularism debate
plenty of underspecified solutions to the Gettier problem
Also, I think I haven’t seen an AF researcher who just relies on a hodge-podge of other researcher’s ideas without questioning them… The researchers I listed above don’t do it.