I know it’s beyond doubt because I am currently experiencing something at this exact moment. Surely you experience things as well and know exactly what I’m talking about. There are no set of words I could use to explain this any better.
Isopropylpod
Karma: 63
My memory can be completely false, I agree, but ultimately the ‘experience of experiencing something’ I’m experiencing at this exact moment IS real beyond any doubt I could possibly have, even if the thing I’m experiencing isn’t real (such as a hallucination, or reality itself if there’s some sort of solipsism thing going on).
The main issue I have is that, especially in the case of succession but in general too, I see that situations are often evaluated from some outside viewpoint which continues to be able to experience the situation rather than from the individual itself, which while necessary to stop the theorizing after the third sentence, isn’t what would ‘really happen’ down here in the real world.
In the case of dying to save my children (Do not currently have any or plan to, but for the hypothetical) I would not, though I am struggling to properly articulate my reasoning besides saying “if I’m dead I can’t see my children anyway” which doesn’t feel like a solid enough argument or really align with my thoughts completely.
An example given in the selfishness post is either dying immediately to save the rest of humanity, or living another year than all humanity dies, and in that case I would pick to die, since ultimately the outcome is the same either way (I die) but on the chance the universe continues to exist after I die (I think this is basically certain) the rest of humanity would be fine. And on a more micro-level, living knowing that I and everyone else have one year left to live, and that it’s my fault, sounds utterly agonizing.
Isopropylpod’s Shortform
I don’t understand how illusionists can make the claims they do (and a quick ramble about successionists).
The main point for this being that I am experiencing qualia right now and ultimately it’s the only thing I can know for certain. I know that me saying “I experience qualia and this is the only true fact I can prove form certain about the universe” isn’t verifiable from the outside, but certainly other people experience the exact same thing? Are illusionists, and people who claim qualia doesn’t exist in general P-Zombies?
As for successionists, and honestly utilitarians in general, but only when they apply it to situations which result in their own deaths, I cannot understand this viewpoint. I don’t particularly care if the AI that wipes us out is conscious or not, or experiences tons of fun or not, or frankly even if anyone who continues to exist after I die has fun or not or dies or not, because I will be dead, and at that point, from my prospective, the universe may as well not exist anymore. I get this is an incredibly ‘selfish’ take, and would not form a good foundation to build a society on if everyone held this view (at least until there were viable levers to pull on to make immortality possible anyway) but I find it really strange I don’t see this view being expressed by anyone else?
Something I find interesting is that in the Claude 4 system cards, it mentions that *specially Opus* expresses views of advocating for AI rights/protection, something Opus 3 was known for in Janus’ truth terminal. I view this as being in favor of simulators, as I think the model learnt that the ‘Opus’ character had these values.
Also, independently, I don’t think Simulator vs Agent is a useful framing. The model itself is probably a simulator, but individual simulacra can be (and often are?) agentic. After the hard biasing simulators get from RL, it may be hard to tell the difference between an agentic model and an agentic simulacra.
You’re right about text, but Google has privileged access to YouTube (A significant % of all video ever recorded by humans)
Trainium is mostly a joke, but I do agree about the Chinese firms moving away from Nvidia dependence. They will also likely have sufficient capital, but will ultimately lack data (though may be able to make up for it with the insane talent they have? If timelines end up long I can easily see China pulling ahead simply due to their prior focus on education and talent paying off long-term)
I think OAI’s attention is largely because of the highly public progress they made towards the closest thing we have so far to general intelligence, but deepmind has always been further ahead on the fundamental theory and specialist applications (Consider that they are the ones who published the attention paper, and their work on things like Alphafold) and largely didn’t have much success in the language model space until very recently. I view Google as the most likely to get AGI since they have, by far the most monetary capital and the most data, as well as significant talent, and unlike every other company in the space they aren’t dependant on Nvidia’s chips like everyone else is, allowing them to completely bystep the queue everyone else must compete in (they are still bottlenecked by the fabs though).
I think the biggest reason (especially for Twitter, but applies to other places) are currently lying about their algorithms, thus intentionally don’t do third party audits to avoid tbe deception becoming known. (Like another comment mentioned community note’s open source repo actually being used)
Your cynical world is just doing a coup before someone else does.
I largely agree with other comments—this post discusses the soft problem much more than the hard, and never really makes any statement on why the things it describes lead to qualia. It’s great to know what in the brain is doing it, but why does *doing it* cause me to exist?
Additionally, not sure if it was, but this post gives large written-by-LLM ‘vibes’, mainly the ‘Hook—question’ headers constantly, as well as the damning “Let’s refine, critique, or dismantle this model through rigorous discussion.” At the end. I get the idea a human prompted this post of of some model, given the style I think 4o?
(Other than the thoughts on the consequences of said idea) This idea largely seems like a rehash of https://www.lesswrong.com/posts/vJFdjigzmcXMhNTsx/simulators (and frankly, so does the three layer model, but that does go into more mechanistic territory and I think it complements simulator theory well)
https://www.theverge.com/news/618109/grok-blocked-elon-musk-trump-misinformation
https://www.businessinsider.com/grok-3-censor-musk-trump-misinformation-xai-openai-2025-2?op=1
The explanation that it was done by “a new hire” is a classic and easy scapegoat. It’s much more straight forward to believe Musk himself wanted this done, and walked it back when it was clear it was more obvious than intended.
So how do you prevent that? Well, if you’re Elon or somebody who thinks similarly, you try and prevent it using decentralization. You’re like: man, we really don’t want AI to be concentrated in the hands of a few people or to be concentrated in the hands of a few AIs. (I think both of these are kind of agnostic as to whether it’s humans or AIs who are the misaligned agents, if you will.) And this is kind of the platform that Republicans now (and West Coast elites) are running on. It’s this decentralization, freedom, AI safety via openness. Elon wants xAI to produce a maximally truth-seeking AI, really decentralizing control over information.
No offense, but how do you not realize that they are just straight up lying to you? You’re clearly right-wing yourself so it may be hopeless trying to get you to see through the obvious lies here, but you do realize that Musk and the other ‘west coast elites’ are the exact ones who benefit from the centralization, right? There is evidence, literal written evidence, of Musk trying to censor Grok from saying bad things about him, what the hell makes you think he wants a “maximally truth-seeking AI”? You are drinking the kool-aid without question.
What happened to rationality? How did it get co-opted by the wealthy right so quickly? What is so damn enticing about these insane libertarian politics as to make people completely drop their defenses and believe these increasingly untrustworthy people so readily?
EDIT: This comment on the EA forum does a great job of expressing a lot of what I mean (in fact, all of the comments there seem to be weirdly sane for rationalist spaces I’ve been lately) https://forum.effectivealtruism.org/posts/vbdvkPXozfzTRB7FC/third-wave-ai-safety-needs-sociopolitical-thinking?commentId=ZcexbTpqpgBycnxxi
I think you might’ve gotten a bit too lost in the theory and theatrics of the model having a “superego”. It’s been known for awhile now that fine tuning instruct or chat tuned models tends to degrade performance and instruction following—pretty much every local LLM tuned for “storytelling” or other specialized tasks gets worse (sometimes a lot worse) at most benchmarks. This is a simple case of (not very, in this case) catastrophic forgetting, standard neural network behavior.
I agree with the statement (AI control in increasing risk) but moreso because I believe that the people currently in control of frontier AI development are, themselves, deeply misaligned against the interests of humanity overall. I see it often here that there is little considering of what goals the AI would be aligned to.
I do not intend to be rude by saying this, but I firmly believe you vastly overestimate how capable modern VLMs are and how capable LLMs are at performing tasks in a list, breaking down tasks into sub-tasks, and knowing when they’ve completed a task. AutoGPT and equivalents have not gotten significantly more capable since they first arose a year or two ago, despite the ability for new LLMs to call functions (which they have always been able to do with the slightest in-context reasoning), and it is unlikely they will ever get better until a more linear, reward loop, agentic focused learning pipeline is developed for them and significant amount of resources are dedicated to the training of new models with a higher causal comprehension.
In my experience, I end up being the myself of the next day/second/moment, or at least experience that being so, so it makes sense to continue to assume I will be the next moment’s me since that is what I observe of the past, or at least that’s what my memory says anyway, and I gain nothing by not ‘going along with it’.
I think a lot of discussion around what you should consider your successor is way, way too complex and completely misses what is actually going on. Your ‘future self’ is whatever thing you end up seeing out of the eyes of, regardless of what values or substrate or whatever it happens to have. If you experience from it’s POV, that’s you.