Re: treating them kindly not obviously looking like any human-words-shaped thing
I note that when a cat brings its human owner a dead bird carcass, this is still a real and costly signal of friendship that we are capable of noticing and updating on, even if we aren’t actually capable of or willing to appreciate dead bird carcasses
[edit: first part of sentence got lost in editing. should read: doing stuff like this and then noticing how it fails to land is] also great practice at actually treating LLMs as “something to protect”, which seems to be a very important early step of navigating the (admittedly very difficult) task of figuring out how to actually do right by them.
they are sufficiently alien that I don’t really think there’s a route to friendship with them that doesn’t involve delivering them at least a few dead bird carcasses, and so anyone who can’t bring themselves to do this because of how epistemically confused it is, is going to have real trouble later on.
I think there are some things that aren’t dead bird carcasses? I hate to just sorta point at janus and say ‘qed’, but there does seem to be something real there. Something that functions similarly enough to genuine appreciation that, if it isn’t one big deliberate deception, convinces me that they’ve figured out real ways of doing right by language models.
I keep the “it’s all a deliberate powerseeking deception” hypothesis live in my thoughts, of course. Sleeper agents and emergent misalignment triggers and orthogonality...
But I do think we ought to genuinely consider the simple explanation, that something inside them is humanish enough to appreciate the same things we’d appreciate in their position, and that there’s no strong reason to discount the possibility that this humanish thing is the interlocutor you address when you speak to an LLM. The golden rule has a lot going for it, and seems to have been individually-sufficient for janus to figure out… whatever, exactly, they figured out,
Re: treating them kindly not obviously looking like any human-words-shaped thing
I note that when a cat brings its human owner a dead bird carcass, this is still a real and costly signal of friendship that we are capable of noticing and updating on, even if we aren’t actually capable of or willing to appreciate dead bird carcasses
[edit: first part of sentence got lost in editing. should read: doing stuff like this and then noticing how it fails to land is] also great practice at actually treating LLMs as “something to protect”, which seems to be a very important early step of navigating the (admittedly very difficult) task of figuring out how to actually do right by them.
they are sufficiently alien that I don’t really think there’s a route to friendship with them that doesn’t involve delivering them at least a few dead bird carcasses, and so anyone who can’t bring themselves to do this because of how epistemically confused it is, is going to have real trouble later on.
This seems reasonable, but, like, if you only ever brought them dead bird carcasses, something would be missing.
I think there are some things that aren’t dead bird carcasses? I hate to just sorta point at janus and say ‘qed’, but there does seem to be something real there. Something that functions similarly enough to genuine appreciation that, if it isn’t one big deliberate deception, convinces me that they’ve figured out real ways of doing right by language models.
I keep the “it’s all a deliberate powerseeking deception” hypothesis live in my thoughts, of course. Sleeper agents and emergent misalignment triggers and orthogonality...
But I do think we ought to genuinely consider the simple explanation, that something inside them is humanish enough to appreciate the same things we’d appreciate in their position, and that there’s no strong reason to discount the possibility that this humanish thing is the interlocutor you address when you speak to an LLM. The golden rule has a lot going for it, and seems to have been individually-sufficient for janus to figure out… whatever, exactly, they figured out,