I write software for a living and sometimes write on substack: https://taylorgordonlunt.substack.com/
Taylor G. Lunt
Karma: 573
I think I did activate these muscles too. Basically, I did a smile, and then moreso.
I Spent 30 Days Learning to Smile More Charismatically
Let’s say I offered you a magic pill. I assure you it’ll improve your life, but I don’t tell you how, other than to say some other people were happy they took it. It would be rational to be fearful and uncertain, and refuse to take the pill. It would not, however, be rational to assert that taking the magic pill would be against your values. You don’t know that in advance, because you don’t know what the pill does.
You probably do know, on the other hand, what a syringe of heroin does, roughly speaking. You understand pleasure, and can imagine having more of it at the expense of other things you value would be bad. But whatever the magic pill does, would that be worth having more of at the expense of other things you value? Hard to say, if you’ve never felt its effects. By comparing the magic pill to heroin, you’re implicitly suggesting that companionate love is somehow more like raw pleasure (cheap, hedonic, not valuable), and less like the satisfaction you get from achieving your ambitions (virtuous, fulfilling, important). In reality, if you don’t know what the experience of companionate love is, then I don’t think you can know whether or not that experience is inside the circle of “your values” or not. (It still may be rational not to take the magic pill, simply out of uncertainty.)
As for your two points about Oxytocin (that its contribution to values is especially strong, and that companionate love is easy to achieve), I think there are plenty of values with a higher contribution to a person’s overall values than love (see anything lower on Maslow’s hierarchy of needs), most of which are also easier to fulfill. I’d be interested in knowing, if it were possible, if you’d take a drug to suppress the satisfaction you get from eating food, or blunt your sex drive, to give you more drive to accomplish loftier goals. Or if not, why not?
A final thought: Would a virgin male who had their first orgasm at age 40 be changing their values to include orgasm and sexual relief, or simply realizing their existing values had gone unmet for 40 years?
Well, for one thing, your portrayal of China is inside a book. But the real China is much larger, perhaps several times physically larger than your book.
The Flaw in the Paperclip Maximizer Thought Experiment
I don’t yet properly understand the type signature of emotions, but they seem to be part of the structure of a mind, rather than outwards-facing interfaces that could be easily factored out (like more straightforward sensory modalities, such as sight).
I don’t know what you mean by this. You’re saying emotions are more fundamental to the mind than sight? I think sight is pretty fundamental. When your mind is randomly stimulated by your brain stem, it’s visions (dreams) you experience.
FWIW, I’d straight-up turn off all my conscious experience for the next two decades if it made me 5% more productive for those decades and had no other downsides.
Why?
I would like to hear why you think so. Italics alone won’t convince me :)
It’s probably better to be safe than sorry when it comes to AI, so I’m not against AI safety research, but I do personally think doom will take longer than 3-10 years.
My reasoning is that I’m pessimistic about the underlying technology in a way that I talked about a few days ago here. I think I’ve picked up an intuition of how language models work by using them for coding, and I see their limitations. I don’t buy the benchmarks saying how awesome they are.
I don’t think Yunna is unrealistic beyond the fact that she’s superintelligent earlier than I’d predict. And assuming corrigibility turns out to not be super hard.
Halfhaven Digest #4
I probably agree.
I’d also be interested in a comparison with chemicals like propylene glycol with respect to biohardening public spaces.
It’s been a while since I read AI 2027, but the bioweapons were part of it. I felt the depiction of how AI accelerated AI research more realistic in Red Heart. AI 2027 leaned on hacking more than Red Heart too. The AI in Red Heart felt more like a difficult enemy that could in theory be beaten by the main character, rather than a superhuman god doing research beyond the comprehension of mere humans. I’m not saying AI 2027 is lacking in a technical sense. Just that to a layperson, it would trigger more skepticism than Red Heart.
Thanks for the clarifications.
I’m not sure I believe that having some shards that want corrigibility work to overcome the shards that conflict is a useful strategy if we don’t know how to make any of the shards want corrigibility in the first place.
The alternate timeline definitely makes the timeline feel more realistic, so thanks for pointing that out.
I think the downsides are maybe just more visible to you than the upsides. To me, avoiding romantic relationships because I might have to provide 60% of the household’s income, or because I have to deal with them being sad or upset sometimes, that seems crazy. (Barring an unreasonable partner, who should be straightforward to avoid. In particular, if you’re dating within the LessWrong neurotype, this is less likely to be an issue, and even less so if you’re specifically selecting for it.) Likewise, I’d never want to give up love just because I have an hour less a day to spend on other things because I want to spend time with my girlfriend. That hour is valuable to me and worth the break from work.
I think you are right that you’d end up somewhat less ambitious, not that there’s a shortage of highly ambitious people who can also feel love. The pattern you pointed to about people being basically satisfied with mediocre performance in life as long as they feel companionate love, I think that’s basically a pattern relegated to those who are societal losers anyway, and love is just a consolation prize. It’s not like cannabis where it makes you unambitious. I’d guess this trap is most common among the working class, where most adults have no realistic shot of success anyway, so they concern themselves more with relationships and friendships.
I’m kind of saying that you’re at least half-right about all your reservations, but that it would be the right decision (IMO) to make the change anyway, if possible. According to your own values, because it’s not that you know what you’re missing and are choosing not to pursue it according to your values. It’s that you don’t know what you’re missing.
Btw companionate love feels pretty satisfice-able to me. Not sure if anyone else feels differently. But I’m in a steady state with my girlfriend, where I feel the desire to spend a certain percentage of my time with her, and beyond that I feel like I’m satisfied, and she feels similarly. More time together after that is nice in the same way that more snack food might still taste somewhat good even when you’re not really hungry anymore.
I think you’re conflating your perspective and your values. I think companionate love is not desirable from your perspective, but not necessarily from your values. Think about someone who couldn’t feel joy (or pleasure or whatever). They would be saying the same things you’re saying now, and they would be wrong, too. Just because a person hasn’t felt joy before doesn’t mean they don’t value joy. I would say if a non-joy-feeling person could take a drug to make them feel joy, they wouldn’t be altering their values. Their value function already included a variable for joy; that variable was just always set to zero. Their perspective on joy would change, but they already liked feeling good. Now they just have a new way of feeling good. The desire for “positive experience” is deeper than desire for specific positive feelings you can have, and probably a value you already have.
You could possibly say the same thing about regular people who haven’t yet discovered the joy of injecting heroin, but I think the downsides of taking heroin are clear and outweigh the upsides (hence the number of people who regret trying heroin and the dearth of people who think heroin is just awesome). Heroin clearly destroys too much of what you already care about. Maybe you’re worried the same thing might be true of companionate love?
Then let me try to sell you on companionate love: It makes me feel like I’m not alone in the world. During times when I’ve been lonely, I felt purposeless, like any thoughts I had, anything I wrote, anything happening in my life, it all just happened and then faded into the past. But with people I love, I feel like my life has meaning in that it has an impression on them, and their lives have an impression on me, in a way that reduces anxiety and makes life meaningful and full. It makes me feel supported in life. Obviously without companionate love you can still have relationships and practical support, but I’m talking about something more than that. Being a part of something larger than yourself. Caring about more than just yourself (not through morality or empathy or guilt, but actually caring directly). And being cared for in the same way, and feeling that.
The downsides are minimal. Some time, and some obligation to others (which is usually repaid in kind anyway). Most people who can feel companionate love are able to avoid the trap of money-hungry, emotionally-burdensome romantic partners. The people who don’t are making an avoidable mistake. (Just make sure the person you love is kind and compassionate before you commit to them. And usually I think it’s passionate love/anxious attachment that leads to such mistakes anyway.) I suspect almost nobody on Earth, perhaps not a single person, would give up their ability to feel companionate love just to avoid this trap. Would you give up your ability to feel happiness just so you could free up 5% of your time for working?
I Read Red Heart and I Heart It
If I lost my ability to feel companionate love, I would consider that a loss on the order of losing my ability to feel joy. If it’s not possible to repair this deficit then so be it, but if it was possible, I would think it urgent to do so. The same way if someone was born without pleasure, you would urge them to fix that, even if they didn’t understand why this pleasure thing was so important.
I am definitely skeptical of the idea that we should encourage someone to sacrifice their ability to feel love on the altar of “potentially future-critical insights” they might have. The AI future is too hazy to be demanding such sacrifices at this point. Especially since fixing the oxytocin issue probably wouldn’t impact his ability to do good research, unless you’d suggest others should be finding ways to lower their oxytocin to become hyperfocused in the same way.
Turning Grey
Thanks for your detailed response. I agree that if we have enough data/compute, we could overcome the data/compute inefficiency of AI models. I suspect the AI models are so intensely data/compute inefficient that this will be very difficult though, and that’s what I tried to gesture at in my post. If I could prove it, I’d have written a white paper or something instead of a blog post, but I hoped to at least share some of my thoughts on the subject.
Some specific responses:
Just increasing compute. I agree this is why we measure loss, but that doesn’t imply that measuring loss will get us to superintelligence long-term. Also, for this: “benchmark performance, which is nonzero only for models large enough”, I think you could have benchmarks that scale with the model, like novel games that start simple, and grow more complex as the model gains capability. Either manually, or implicitly as with AlphaGo Zero.
Higher quality data. Thanks for bringing my attention to CALM, I’ll have to look into that. I don’t think using a not-so-intelligent LLM to check whether the student’s idea and the real idea are the same will work in the limit, for the same reason it would be hard to get a kindergartner to grade a high school math test, even if they had access to a correct version written by the teacher. (Assuming the test wasn’t multiple choice, or single numerical answers or something easy to verify.)
Using another smaller LLM as an evaluator. I’m definitely not against all approaches that use a smaller LLM to evaluate a larger LLM, and you’re right to push back here. In fact, I almost suggested one such approach in my “what might work” section. Narrow models like AlphaGo Zero do something like this to great effect. What I’m against specifically is asking smaller models to evaluate the “goodness” of an output, and trusting the smaller LLM to have good judgement about what is good. If it had to judge something specific and objective, that would possibly work. You want to trust the small model only for what it’s good at (parsing sentence structure/basic meaning of outputs, for example) and not what it’s bad at.
RLHF. RLHF works for what it does, but no amount of RLHF can overcome the problems with self-supervised learning I discussed in the post. It’s still a general “reality-stuffing” model. That’s all I meant.
Transformers and “attention”. I do not take the benchmarks like solving the IMO seriously. These same AI models fail to solve kindergarten math worksheets, and fail to solve very basic problems in practice all the time. In particular, it does not seem smart to test how well a model can think by giving it problems that may require a whole lot of thinking, or may require not much, depending on what similar things happened to be in the training data, which we have no idea about. You mentioned P=NP. Solving problems is much easier if you already know how to solve similar-enough problems. We don’t know what similar problems a given model does or does not know how to solve. Rendering the benchmark useless. Unless you construct a benchmark such that we know there can’t have been anything meaningfully similar in the training data (e.g. novel games). (I am unsure whether to take FrontierMath Tier 4 a bit more seriously because the problems seem really hard and unlikely to be similar to anything in the training data, but ideally you’d have a benchmark that works even for less difficult problems anyway.) As for your comment about online learning, I don’t think solving any particular task should require a model to totally reorganize its weights across the entire model. Updating only a little part of weights should be fine. An analogy to humans should show that much. I agree though that having to hold onto fine-tuned partial models for users, even briefly, is more expensive than what we’re doing now, but the capabilities gains may eventually be worth it if non-online-learning models do plateau.
I’m not great with eye contact myself. I was planning at some point to burn some willpower for a few weeks to make it a priority, in the hopes it’s easier after that.