I do care about my clone as much as myself though—when I look in the mirror, I forget that the reflection isn’t “me” (whatever I am… it seems to fluctuate, is it “my hair” or is that a part of me?). On the other hand sometimes I feel disappointment or displeasure at parts of myself, both mental parts such as traits and behaviors which I personify. And sometimes actual parts—“I hate this pimple”.
That aside, I am struggling to find one decision or specific action which this line of inquiry might change as it feels very much based on abstraction and analogy. I’m particularly confused by this statement:
Maybe we should say: ‘attention is not just all you need, it is all you are’.
Why should we say that? Are A.I. researchers really walking around, peering over someone slaving away at there terminal when they are having a problem with the kinds of responses an A.I. is generating and with a smirk reminding them “attention is all you need”. What exactly is meriting a change to ”...attention is all you are”? Can you make this a little more literal and concrete for my unpoetic mind?
I was being a little hyperbolic but I guess the point of “attention is all you are” was to say that the way in which you are different from your clone is that you have a different context to them. One AI instance is a different entity to another AI instance because it has a different context: a different KV cache ⇔ a different entity. In other words, your KV cache and query vectors (your attention) literally defines who you are.
the way in which you are different from your clone is that you have a different context to them.
Can you maybe use a different word than “context” because it feels very imprecise and anything can be “context”. But I’m not sure how it defines identity. And I will have a future context in the future, and I’ve had a different context in the past. But I’m rarely “oh screw that guy” unless I know that the context, attitudes, or expectations I’ve had in the past were particularly irrational.
In other words, your KV cache and query vectors (your attention) literally defines who you are.
I’m unconvinced. Again, can you establish why context defines identity...? Assuming that you can, well, when I show “care” for who I will be in the future, I’m often entertaining a range of contexts. Hedging, “I don’t know how I’ll feel about it tomorrow so I’ll leave my options open.”
To put this by analogy to a LLM or a Diffusion Model, there are a finite number of contexts based on which tokens activate which weights. It feels like we need to use Emo Phillips joke about which denomination you belong to to determine what distinguishes one identity from another. At which point—why bother?
Firstly, when can you safely assume that a model has a “identity”? The most recent research I’ve seen is that LLMs can’t accurately predict their own reasoning[1] and only for a short amount of steps—but now I sound like I’m making it up because I can’t find said research in a timely manner. Of course LLM or AI ability to self-model accurately could improve in the future—but what will be the specific details of that change?
Note—I’m making the assumption that “identity” is really a matter of a model of an entity’s own behavior, habits, and yes—external often social markers like “Northern Conservative Baptist or Southern Conservative Baptist?” “Manchester United or Chelsea?” “Olivia Rodrigo or Taylor Swift” “Beatles or Stones?” “MJ or Prince?” “Arnie or Sly?”—self-awareness is basically the ability to accurately predict one’s own responses or identify revealed preferences—hopefully without succumbing to any negative self-fulfilling feedback loops.
https://link.springer.com/chapter/10.1007/978-3-031-78977-9_3 ″they do not fully and accurately follow the model’s decision process, indicating a gap between perceived and actual model reasoning.” that being said… “We show that this gap can be bridged because prompting LLMs for counterfactual explanations can produce faithful, informative, and easy-to-verify results.”
In the clone thought experiment, ‘context’ just refers to all of the sensory inputs you have ever received and all of the thoughts you have ever had. For a LLM instance, it just refers to the KV cache. Since you are identical to your clone except for differences in context since the cloning took place, this context is a defining part of who ‘you’ are. But yes, I am being overly zealous when I say that this defines you—it is better to say that your context is a part of who you are, which is not really a very novel statement.
I do agree that we care about our future self (who will have a different context), and we would care about our clone—just usually both to a lesser extent than we care about our current self. Interestingly, I think I would care more about my future self than I would care about my clone, even if the clone had a greater percentage of shared history.
I do care about my clone as much as myself though—when I look in the mirror, I forget that the reflection isn’t “me” (whatever I am… it seems to fluctuate, is it “my hair” or is that a part of me?). On the other hand sometimes I feel disappointment or displeasure at parts of myself, both mental parts such as traits and behaviors which I personify. And sometimes actual parts—“I hate this pimple”.
That aside, I am struggling to find one decision or specific action which this line of inquiry might change as it feels very much based on abstraction and analogy. I’m particularly confused by this statement:
Why should we say that? Are A.I. researchers really walking around, peering over someone slaving away at there terminal when they are having a problem with the kinds of responses an A.I. is generating and with a smirk reminding them “attention is all you need”. What exactly is meriting a change to ”...attention is all you are”? Can you make this a little more literal and concrete for my unpoetic mind?
I was being a little hyperbolic but I guess the point of “attention is all you are” was to say that the way in which you are different from your clone is that you have a different context to them. One AI instance is a different entity to another AI instance because it has a different context: a different KV cache ⇔ a different entity. In other words, your KV cache and query vectors (your attention) literally defines who you are.
Can you maybe use a different word than “context” because it feels very imprecise and anything can be “context”. But I’m not sure how it defines identity.
And I will have a future context in the future, and I’ve had a different context in the past. But I’m rarely “oh screw that guy” unless I know that the context, attitudes, or expectations I’ve had in the past were particularly irrational.
I’m unconvinced. Again, can you establish why context defines identity...? Assuming that you can, well, when I show “care” for who I will be in the future, I’m often entertaining a range of contexts. Hedging, “I don’t know how I’ll feel about it tomorrow so I’ll leave my options open.”
To put this by analogy to a LLM or a Diffusion Model, there are a finite number of contexts based on which tokens activate which weights. It feels like we need to use Emo Phillips joke about which denomination you belong to to determine what distinguishes one identity from another. At which point—why bother?
Firstly, when can you safely assume that a model has a “identity”? The most recent research I’ve seen is that LLMs can’t accurately predict their own reasoning[1] and only for a short amount of steps—but now I sound like I’m making it up because I can’t find said research in a timely manner. Of course LLM or AI ability to self-model accurately could improve in the future—but what will be the specific details of that change?
Note—I’m making the assumption that “identity” is really a matter of a model of an entity’s own behavior, habits, and yes—external often social markers like “Northern Conservative Baptist or Southern Conservative Baptist?” “Manchester United or Chelsea?” “Olivia Rodrigo or Taylor Swift” “Beatles or Stones?” “MJ or Prince?” “Arnie or Sly?”—self-awareness is basically the ability to accurately predict one’s own responses or identify revealed preferences—hopefully without succumbing to any negative self-fulfilling feedback loops.
https://link.springer.com/chapter/10.1007/978-3-031-78977-9_3
″they do not fully and accurately follow the model’s decision process, indicating a gap between perceived and actual model reasoning.” that being said… “We show that this gap can be bridged because prompting LLMs for counterfactual explanations can produce faithful, informative, and easy-to-verify results.”
In the clone thought experiment, ‘context’ just refers to all of the sensory inputs you have ever received and all of the thoughts you have ever had. For a LLM instance, it just refers to the KV cache. Since you are identical to your clone except for differences in context since the cloning took place, this context is a defining part of who ‘you’ are. But yes, I am being overly zealous when I say that this defines you—it is better to say that your context is a part of who you are, which is not really a very novel statement.
I do agree that we care about our future self (who will have a different context), and we would care about our clone—just usually both to a lesser extent than we care about our current self. Interestingly, I think I would care more about my future self than I would care about my clone, even if the clone had a greater percentage of shared history.