Jessica Taylor. CS undergrad and Master’s at Stanford; former research fellow at MIRI.
I work on decision theory, social epistemology, strategy, naturalized agency, mathematical foundations, decentralized networking systems and applications, theory of mind, and functional programming languages.
Blog: unstableontology.com
Twitter: https://twitter.com/jessi_cata
The first thing I can remember is that I learned at age 3 that I would die someday, and I cried about it. I got my hopes up about radical technological progress (including AGI and biotech) extending lifespan as a teenager, and I lost most of that hope (and cried the most I ever had in my life) upon realizing that AGI probably wouldn’t save us during our lifetime, alignment was too hard.
In some sense this outcome isn’t worse than what I thought was fated at age 3, though. I mean, if AGI comes too soon, then I and my children (if I have them) won’t have the 70-80 year lifetimes I expected, which would be disappointing; I don’t think AGI is particularly likely to be developed before my children die, however (minority opinion around here, I know). There’s still some significant chance of radical life extension and cognitive augmentation from biotech assisted by narrow AI (if AGI is sufficiently hard, which I think it is, though I’m not confident). And as I expressed in another comment, there would be positive things about being replaced by a computationally massive superintelligence solving intellectual problems beyond my comprehension; I think that would comfort me if I were in the process of dying, although I haven’t tested this empirically.
Since having my cosmic hopes deflated, I have narrowed the scope of my optimization to more human-sized problems, like creating scalable decentralized currency/contract/communication systems, creating a paradigm shift in philosophy by re-framing and solving existing problems, getting together an excellent network of intellectual collaborators that can solve and re-frame outstanding problems and teach the next generation, and so on; these are still ambitious, and could possibly chain to harder goals (like alignment) in the future, but are more historically precedented than AGI or AI alignment. And so I appear to be very pessimistic (about my problem-solving ability) compared to when I thought I might be able to ~singlehandedly solve alignment, but still very optimistic compared to what could be expected of the average human.
I don’t remember the exact words in our last conversation. If I said that, I was wrong and I apologize.
Ok, the opinions you’ve described here seem much more reasonable than what I remember, thanks for clarifying.
I do think that psychosis should be thought of differently than just “weird thoughts that might be true”, since it’s a whole-body nerve-and-brain dysregulation of which weird thoughts are just one symptom.
I agree, yes. I think what I was afraid of at the time was being called crazy and possibly institutionalized for thinking somewhat weird thoughts that people would refuse to engage with, and showing some signs of anxiety/distress that were in some ways a reaction to my actual situation. By the time I was losing sleep etc, things were quite different at a physiological level and it made sense to treat the situation as a psychiatric emergency.
If you can show someone that they’re making errors that correspond to symptoms of mild psychosis, then telling them that and suggesting corresponding therapies to help with the underlying problem seems pretty reasonable.
Thanks, I appreciate you saying that you’re sorry my experience was bad towards the end (I notice it actually makes me feel better about the situation), that you’re aware of how criticizing people the wrong way can discourage speech and are correcting for that, and that you’re still concerned enough about misconstruals to correct them where you see fit. I’ve edited the relevant section of the OP to link to this comment. I’m glad I had a chance to work with you even if things got really confusing towards the end.
I want to point out that the level of mental influence being attributed to Michael in this comment and others (e.g. that he’s “causing psychotic breaks” and “jailbreaking people” through conversation, “that listening too much to Vassar [causes psychosis], predictably”) isn’t obviously less than the level of mental influence Leverage attributed to people in terms of e.g. mental objects. Some people in the thread are self-congratulating on the rationalists not being as crazy and abusive as Leverage was in worrying that people were spreading harmful psychological objects to each other, and therefore isolating these people from their friends. Yet many in this comment thread are, literally, calling for isolating Michael Vassar from his friends on the basis of his mental influence on others.