Mikhail Samin

Karma: 3,469

My name is Mikhail Samin (@Mihonarium on Twitter/X, @misha on Telegram).

Humanity’s future can be enormous and awesome; losing it would mean our lightcone (and maybe the universe) losing most of its potential value.

I have takes on what seems to me to be the very obvious shallow stuff about the technical AI notkilleveryoneism; but many AI Safety researchers told me our conversations improved their understanding of the alignment problem.

I’m running two small nonprofits: AI Governance and Safety Institute and AI Safety and Governance Fund. Learn more about our results and donate: aisgf.us/fundraising

I took the Giving What We Can pledge to donate at least 10% of my income for the rest of my life or until the day I retire (why?).

In the past, I’ve launched the most funded crowdfunding campaign in the history of Russia (it was to print HPMOR! we printed 21 000 copies =63k books) and founded audd.io, which allowed me to donate >$100k to EA causes, including >$60k to MIRI.

[Less important: I’ve also started a project to translate 80,000 Hours, a career guide that helps to find a fulfilling career that does good, into Russian. The impact and the effectiveness aside, for a year, I was the head of the Russian Pastafarian Church: a movement claiming to be a parody religion, with 200 000 members in Russia at the time, trying to increase separation between religious organisations and the state. I was a political activist and a human rights advocate. I studied relevant Russian and international law and wrote appeals that won cases against the Russian government in courts; I was able to protect people from unlawful police action. I co-founded the Moscow branch of the “Vesna” democratic movement, coordinated election observers in a Moscow district, wrote dissenting opinions for members of electoral commissions, helped Navalny’s Anti-Corruption Foundation, helped Telegram with internet censorship circumvention, and participated in and organized protests and campaigns. The large-scale goal was to build a civil society and turn Russia into a democracy through nonviolent resistance. This goal wasn’t achieved, but some of the more local campaigns were successful. That felt important and was also mostly fun- except for being detained by the police. I think it’s likely the Russian authorities would imprison me if I ever visit Russia.]

Mikhail Samin 26 May 2026 11:55 UTC
2 points
0
in reply to: jimrandomh’s comment on: There should be a discussion about LW’s policy to allow calls for violence
Thanks! I think I mostly agree with what you’re optimizing for. Some comments:
On why delete: I’m not particularly worried about people convincing LW users to commit violence. I’m worried about people who are much more willing to commit violence than approximately all LW users finding each other via LW.
On evidence of common knowledge: as I mentioned in the post, people would expect self-censorship, especially from senior community members, and so I’d be worried about people still incorrectly concluding that others are secretly more sympathetic to violence than they outwardly appear, even in the absence of rules prohibiting specific calls for violence.
On preserving evidence: I’m very sympathetic to it, and also very sympathetic to having police officers wear body-cams. I think this does not require the comments to exist under schelling-pointy posts about violence, though? E.g., one way to do this would be to remove a comment from the post, but keep it available (or even actively reported) to law enforcement, and available to other users (established users?) in the moderation log.
On speaking with a lawyer: I think automod already somewhat prevents people from speaking further without their lawyer; but also I’m not sure how to best balance being a place where people can incriminate themselves and preventing people like that from finding collaborators using the platform. (Or are you worried that established users would err on the side of speaking with a lawyer even when not actually warranted?)

Mikhail Samin 26 May 2026 11:33 UTC
10 points
0
in reply to: habryka’s comment on: There should be a discussion about LW’s policy to allow calls for violence
Thank you. It appears that I have indeed misunderstood your position. I apologize for that and I apologize for the fallout.
There are clearly some things that would be over the line. If someone posts a comment being like “I will show up to <company office X> tomorrow and firebomb them, show up if you read this and want to participate”, I would very likely take it down (and also report it to the police and share what info I have on who wrote it)
Very happy to hear this.
My two moderation comments on the issue do not read to me as implying this kind of dichotomy.
To clarify why I interpreted them this way: a comment mentioned “discussing or advocating violence is not banned on LessWrong”, with the only mentioned exception to that being only the usual moderation rules^[1]. I expected that the general policy is “to permit calls for violence on the site, with the expectation that they will get heavily downvoted^[2], and that this will be better at deterring future people from making similar calls”, without exceptions that depend on how broad the targets of calls for violence are.
we e.g. deactivated his ability to send DMs on LessWrong
Apologies, I did not have this context^[3]. Thank you.
I’m curious why this particular comment was not over the line/wasn’t deleted, if you want to share. It did not strike me as advocating violence only in general terms, and people could easily get in touch with the author by looking at the contact details on his profile.
...if a group of volunteers was starting to coordinate how to join and assist the Ukraine war
...People should be able to discuss whether or not to commit violence against large groups of people, like asking whether now is the time to have a revolution against the government
… Using LessWrong to actually coordinate or execute plans… if somehow the people doing that had gotten community buy-in on that being a good idea
Yep, I agree all of those should be fine on LessWrong (and actually I think they’re all fine on Twitter or Telegram).
I think it’s would be good to not allow using LessWrong as a platform for finding people who want to coordinate and execute plans of this type for which there isn’t a community buy-in, even when the coordination itself happens outside LessWrong.
that user’s bio they currently have written that they are actively looking for “cofounders” to do some cybercrime stuff, and encouragement to reach out to them in private, and my guess is that crosses my line for what I want people to do on LessWrong
Also great to hear this.^[4]
1. ^
  [that still apply if someone’s being a dick or making the discussion go off the rails]
2. ^
  “Heavily downvoted” is also the language I used in the tweet; I hope you can see that it wasn’t transparent to me that this would not work describe the general situation with the karma of such content when I made the tweet.
3. ^
  I did not expect this, given I reported his DMs but wasn’t told of any action taken.
4. ^
  (Relatedly, my impression is that a goal of his cybercrime ideas is to doxx people to enable killing them.)

Mikhail Samin 25 May 2026 22:40 UTC
5 points
2
in reply to: Brendan Long’s comment on: There should be a discussion about LW’s policy to allow calls for violence
by posting an out-of-context screenshot with the karma removed implying that LessWrong supports violence
The tweet literally said “highly downvoted” And was not in any way “out-of-context”. It did not imply LessWrong supports violence; I’ve made many comments explaining that the opposite is the case.

There should be a discussion about LW’s policy to allow calls for violence

Mikhail Samin25 May 2026 13:51 UTC

−5 points

21 comments10 min readLW link

Mikhail Samin 21 May 2026 20:39 UTC
2 points
0
in reply to: Viliam’s comment on: Mikhail Samin’s Shortform
I think most of the issue was with it being posted on Twitter (later removed). The audience outside LW has no clue what the second karma-looking-number is.
I have not linked to the comment, and I think most of the issue was with it was easy to contact the author of the comment until they deactivated their account, not as much with the actual call (as it was for people to organize to do crime, not to do specific actions right now and alone).

Mikhail Samin 16 May 2026 0:52 UTC
2 points
0
in reply to: habryka’s comment on: Mikhail Samin’s Shortform
I did link to it in the original version of my comment and did not have a screenshot attached at all. (Feel free to look up the full edit history of the comment).
I later replaced the comment with an approximate copy of tweet (mostly for consistency and seeing that many of lw users liked it).
Why I did not include an uncropped screenshot in the tweet:
The name of the person would’ve been visible and it would be much harder to not make it dog-whistly and to not make people who want violence easily able to contact the person, and also it had irrelevant parts at the beginning, and I didn’t want to redact it because I was lazy (it’s tricky on iPhone) and didn’t want it to look like I’m protecting the identity of the person. So I simply cropped to the relevant part and said that it’s “heavily downvoted”. Idk what −68 karma on the screenshot would’ve communicated that “heavily downvoted” didn’t.
I did explicitly want to include “heavily downvoted” to reduce the chance of anyone possibly thinking that the LessWrong community agrees with what the comment suggests (since the crop made the karma invisible), and I spent a lot of time replying to people on Twitter who tried to say that the call for violence is downstream of anything LW is. I also pointed out to many people that the text of the comment is not visible by default due to it being heavily downvoted. (Maybe wrote a couple dozen comments in total defending LW?)
I was not aware that this is a policy that you already have and discussed with others; I would not have taken it to Twitter, in this specific form, with the goal that I had.

Mikhail Samin 15 May 2026 10:06 UTC
−1 points
0
on: Mikhail Samin’s Shortform
Why is LessWrong allowing calls for assassinations?

They’re heavily downvoted, but not removed and the people expressing them are not banned.

Mikhail Samin 5 May 2026 18:58 UTC
5 points
0
in reply to: Linch’s comment on: Mikhail Samin’s Shortform
I didn’t have to memorize them! just make them all basically the same; then find the one that was different due to the 180* rotation

Mikhail Samin 5 May 2026 11:29 UTC
6 points
0
on: Mikhail Samin’s Shortform
Apparently failing SAN checks when interacting with me can make someone think I’m literally Omega????
After a while after game of poker, I shuffled a deck of cards that came with a poker set. I then, wanting it to be as impressive as possible, gave the deck to someone who has a background in non-existential bad risks and acausal trade to make him go through all of the steps himself, with his hands. I told him to shuffle it; he did a bunch of shuffling. I then told him to face the deck and go through it, stopping at any point (like when a real magician does it to you and you say stop, but on his own) and freely picking a card (while seeing them). I closed my eyes shut and turned away while he was doing this.
He was done, I turned back and told him to pull out the card that he wanted and put it in his pocket, then shuffle the rest of the cards. He shuffled. I told him there’s only like 50% chance the trick works. I stepped closer and briefly touched the cards. I then stepped back so I couldn’t really see anything and told him to put the cards back into the deck and shuffle it again. He did that. I then asked him to give the deck back to me. I shuffled the deck some more, then went through the deck, without looking at the front of the cards.
His card was on the very top of the deck; I pointedly flashed it to him and asked him what card was his. He told me it was the king of hearts. I gave the card.
He was pretty amazed; there was no way I could’ve done this that he could think of, except me somehow feeling the heat of the pocket of his hoodie left on the card.
I then asked him how surprised he would be if I made an advanced prediction of which card he would pick. I then asked him to pull out my phone, unlocked it, told him to open claude (he didn’t know how to do this on iOS so I did that), asked him to open the list of chats and go to the bottom two about poems.
Both had pictures of me holding a deck of cards with a king of hearts and asking Claude to make a poem about that/about a magic trick where the secret card is the king of hearts.
At this point, the best explanation was that I’m literally Omega and he is in a simulation.
(Oops?)
So, how did I do that???
(Perhaps do not read this if you want to be amazed by it! For a lot of people, explanations somehow destroy the magic of it. Or try to guess, before you read further!)
I don’t really ever do magic. I went to a small poker thing between friends for fun, with £10 buy-in. One of the players was a magician (they were not allowed to deal the cards). They showed us a few card tricks; that was fun, but I saw everything that was going on, with a single exception. After the game, I looked at the cards that came with the poker set (they actually had a nicer feel than the ones we were playing with) and noticed that on the back side, the rhombuses were not cut by the borders of the cards exactly the same way. (If a casino uses cards like that, people can learn to recognize all of the cards and exploit the casino; you really want really precise cutting, or maybe the absence of a pattern near the borders.) What’s more, on most cards, almost all of them cut almost through the rhombuses on one side but left a tiny bit of space with a pattern on the other side.
An idea for a trick immediately came to my mind. Simply rotate all of the cards so the side with rhombuses next to the border is on the left. Then shuffle the deck in front of someone; tell them to pick a card from the deck, make them put it back but rotated 180* (should be pretty simple to achieve — just rotate the rest of the deck somewhat, no one ever pays attention to that stuff; though requires looking closely to make sure it is placed back correctly if they hold it etc.), then shuffle it etc., then find the card and, e.g., take a picture of the card, telling Claude to make a poem about it; or ask the person to think really hard of the card, then take a picture of them and tell Claude to figure out what card they think about (you have to correct Claude with the right card, but then Claude still writes a poem about it).
The very cool thing here is that people have no clue. They just do not suspect that you can somehow be looking at cues on the backside of the cards; it’s supposed to be impenetrable. I think I actually tried to hard to not appear like I’m looking at the cards or doing much while going through their borders: no one actually paid attention, people do not get sus about anything at this point, the card is supposed to be completely lost. I actually asked them about what could I possibly have done, and then took steps to make sure it was impossible for me to use the thing they assumed I could. (I wouldn’t actually be able to, for most of the ideas; I can track some sleight of hand, but I don’t have skills to do any of it.)
So, they’re absolutely amazed.
But this time was different.
See, the first time I tried this trick on someone, they (randomly, without looking at the cards before picking it) picked the king of hearts. (I twice took pictures of it and asked Claude to come up with poems about it.)
This immediately elevated the trick the last guy experienced to the level of absolutely impossible. I could show him my Claude chats (and I could later show the people around who needed more convincing the timestamps from the Claude on the web that the pictures and the chats were from a few hours prior and these appeared to be clearly unique chats).
So, he was completely stunned: i appeared to make an advance prediction before he looked at the card faces and picked the one he wanted, completely freely, with no pressure from me.
Being someone who spent a lot of time thinking about s-risks (not particularly real), acausal trade, and agents predicting each other, his best explanation was that he is literally in a simulation and I’m literally Omega, the alien philosopher-troll.
While the trick is great, the part where I was able to make an advance prediction is complete luck; there was approximately 2% chance of it happening, but it was absolutely amazing. (Other people also had absolutely no clue how I could possibly have done this. They let me keep the cards!)
Very amusing. Absolutely worth the £10 lost in a few hours of poker.

Mikhail Samin 3 May 2026 13:35 UTC
1 point
0
on: How Go Players Disempower Themselves to AI
It‘s always been strange and sad to me when people aren’t taught to grasp the math and instead learn to just apply rules they can’t rederive to solving problems, so I can empathize with that part of the post.
I don’t really play go, or chess, but at some point, Manifold was trying to play chess against someone, and I tried to play through consequences of various moves with Stockfish (using stockfish was explicitly allowed); anytime stockfish recommended a move I’d try different ones and see if they’re good or not and why; and it was also pretty interesting to explore the tree of chess from the start position with stockfish; I think this gave me some intuitive sense for why some moves are good and some aren’t and I was better at playing chess afterwards.
I’m curious to what extent you think this actually works in chess, and if (and why) you think this doesn’t work in Go.

Mikhail Samin 1 May 2026 19:59 UTC
2 points
0
on: What do Russian academic olympiad winners think of HPMOR? Our data
(correction: claude didn’t deduplicate multiple feedback form submissions from the same people; now fixed.)

Mikhail Samin 1 May 2026 15:48 UTC
2 points
0
on: What do Russian academic olympiad winners think of HPMOR? Our data
Pretty nice numbers, even given that people who liked the book would be more likely to fill out the form

What do Russian olympiad winners think of HPMOR? Our data

Mikhail Samin1 May 2026 13:28 UTC

21 points

2 comments1 min readLW link

Mikhail Samin 26 Apr 2026 12:57 UTC
10 points
1
in reply to: Grendel1209’s comment on: Today might be my last birthday
Sorry, answering quickly with mostly cached thoughts without engaging deeply:
- Current LLM architectures can probably do everything. There are ways of compiling code into transformer’s weights. they’re not good at everything, yet, and generally end up kinda spiky; but it’d be surprising if you couldn’t just scale them up and do a lot more RL and get something pretty general.
- While the 0days discovered by Mythos are all of the kind that could be discovered by humans, humans have in fact looked at some of the code, a lot, manually and using instruments, and failed to find this stuff. I don’t dispute these are not on a level above best human cybersecurity researchers (and have a market on whether there will be any such vulnerabilities discovered by AI that couldn’t have been discovered by humans at all, in the next few years). Being just as good as best humans but much faster is sufficient to take over the world.
- SSI has zero customers, they still get billions of dollars. Google has a lot of money and compute and use AI to develop better chips/compute infra. There are lots of efficiency gains that stack, and some of those are getting automated. I can imagine three main AI companies running out of money, but find that unlikely. The promise of higher intelligence is worth a lot: even if it’s expensive to get to it and to use it, it’s cheaper than paying humans to perform this same tasks, and due to that the demand is enormous.
- I’d not want to bet on AI companies forever being unable to solve jailbreaks sir caring sufficiently about those to not release models at all/ever. In the limit, you do a huge ton of classifiers of texts and activations and report users who try to do bad things to authorities, monitor wetlabs/have honeypot wetlabs that LLMs recommend, make LLMs unaware of some knowledge, etc.; if this really prevents AI companies from releasing and earning tens+ of billions of dollars, they will throw billions of dollars at solving this and will solve this successfully. (E.g., imagine anyone who submits a working jailbreak gets $10k, up to a million submissions each of which gets patches together with similar things; do you think it even gets to a million submissions?)
- ”Alignment“ of current models to not teach people to kill a lot of people mostly has nothing to do with the difficulties we’d face at the superintelligent level.
- What is a thing a transformer architecture cannot do in principle? Like, are you imagining that if we figure out how to make literal superintelligences, something would prevent us from compiling them into transformers? Given in-context learning and scratchpad maintenance are allowed etc.

Mikhail Samin 25 Apr 2026 20:19 UTC
2 points
0
on: “Thinkhaven”
I want to do something like this.

Mikhail Samin 25 Apr 2026 20:17 UTC
3 points
0
in reply to: Jeremy Kalfus’s comment on: Today might be my last birthday
yep, i did not spend enough cycles on the version of the post with this start. Originally it had something like ”This is the first birthday I’m not sure isn’t my last.” but two of three people I asked for feedback told me it sounds kinda suicidal so I changed it at the end. “Might realistically” would’ve represented this better

Today might be my last birthday

Mikhail Samin24 Apr 2026 15:18 UTC

29 points

5 comments5 min readLW link

Mikhail Samin 23 Apr 2026 14:36 UTC
2 points
0
on: An Angry Review of Greg Egan’s “Didicosm”
I think this is not a very well-written piece of fiction, but I think Mitchell’s comment is pretty much right.
I did enjoy the very ending, and the description of the process of science, irrespective of the author’s views on the discussed topics in reality.

Mikhail Samin 23 Apr 2026 12:37 UTC
3 points
0
in reply to: Alex Vermillion’s comment on: Should you donate to Lightcone Infrastructure?
Taking into account your LW profile description: you’re pretty wrong about what were the relevant considerations. I would not particularly think about the consequences of others responding to what I think I should do, as I know this could nudge me towards not doing that, even though I wouldn’t want to be moved by such things; I talked more about my stance towards that here: https://mikhailsamin.substack.com/p/on-access-and-whistleblowing.
(I stopped modelling Lightcone Infrastructure as dath ilani, the fictional version of humanity, due to Oliver’s policy I describe in the post, and I’d publish this post just the same even if I thought Lightcone Infrastructure were going to beat me up or something in response. But mostly considerations like “what if they kick me out” have not crossed my mind because these are irrelevant considerations in the context of deciding to publish a post I think I should.)

Mikhail Samin 20 Apr 2026 18:59 UTC
1 point
−3
in reply to: Anthony DiGiovanni’s comment on: CLR’s Safe Pareto Improvements Research Agenda
- there’s no need to have identical notions of what’s fair in order to cooperate (and no need to have even similar notions in order to cooperate most of the time)
- commitment races are not a real problem, theres no reason to do it, you don’t need to care about logical commitment/decision time in order to incentivize fair splits of gains, and in general, the core of being an FDT agent is being the kind of agent that takes the best actions (and this being predictable about you, without you having to pre-commit to specific actions in specific situations in advance). (I’ve had detailed discussions around this and related stuff with James Faville; some of it was in the format of a debate, and according to people who were around for the debate, I won. one can simply follow the procedure I described in the game of chicken.)
- an agent that incentivizes fair splits ends up in a better situation than most other agents; agents and parts of agents that follow LDTs, including agents that give in to counterfactual mugging are selected for

Mikhail Samin

There should be a dis­cus­sion about LW’s policy to al­low calls for violence

Apparently failing SAN checks when interacting with me can make someone think I’m literally Omega????

What do Rus­sian olympiad win­ners think of HPMOR? Our data

To­day might be my last birthday

There should be a discussion about LW’s policy to allow calls for violence

What do Russian olympiad winners think of HPMOR? Our data

Today might be my last birthday