You are right. I meant that you can’t backprop through it even with an RNN.
You can differentiate it. That’s what you do with LLMs and RL similar to the policy gradient I described above.
You are right. I meant that you can’t backprop through it even with an RNN.
You can differentiate it. That’s what you do with LLMs and RL similar to the policy gradient I described above.