To elaborate somewhat, you could say that the token is the state, but then the transition probability is non-Markovian and all the math gets really hard.
Intuitively I would say that all the tokens in the token window are the state.
And when you run an inference pass, select a token and append that to the token window, then you have a new state.
The model looks a lot like a collection of nonlinear functions, each of them encoded using every parameter in the model.
Since the model is fixed after training, the only place an evolving state can exist has to be in the tokens, or more specifically the token window that is used as input.
The state seems to contain, for lack of a better word, a lot of entanglement. Likely due to attention heads, and how the nonlinear functions are encoded.
There is another way to view such a system, one that while deeply flawed, at least to me intuits that whatever Microsoft and OpenAI are doing to “align(?)” something like Bing Chat is impossible (at least if the goal is bulletproof).
I would postulate:
- Alignment for such a system is impossible (assuming it has to be bulletproof)
- Impossibility is due to the architecture of such a system
To elaborate somewhat, you could say that the token is the state, but then the transition probability is non-Markovian and all the math gets really hard.
Intuitively I would say that all the tokens in the token window are the state.
And when you run an inference pass, select a token and append that to the token window, then you have a new state.
The model looks a lot like a collection of nonlinear functions, each of them encoded using every parameter in the model.
Since the model is fixed after training, the only place an evolving state can exist has to be in the tokens, or more specifically the token window that is used as input.
The state seems to contain, for lack of a better word, a lot of entanglement. Likely due to attention heads, and how the nonlinear functions are encoded.
There is another way to view such a system, one that while deeply flawed, at least to me intuits that whatever Microsoft and OpenAI are doing to “align(?)” something like Bing Chat is impossible (at least if the goal is bulletproof).
I would postulate:
- Alignment for such a system is impossible (assuming it has to be bulletproof)
- Impossibility is due to the architecture of such a system
I assume that any bit in the input affects the output, and that a change in any parameter has potential impact on that bit.
If anyone want to hear about it, I would be happy to explain my thinking. But be aware the abstraction and mapping I used was very sloppy and ad hoc.