Very interesting and useful! I wonder how much the ceiling from what is ultimately achievable with CoT is determined from the “bag of tricks” available in the forward pass (e.g., knowledge, algorithms embedded in the parameters, “intuitions,” etc.), which is hard to improve without degrading monitorability vs. context length/”how much of the previous context is taken into account” when predicting a single token. I imagine this would be very task-dependent.
Very interesting and useful! I wonder how much the ceiling from what is ultimately achievable with CoT is determined from the “bag of tricks” available in the forward pass (e.g., knowledge, algorithms embedded in the parameters, “intuitions,” etc.), which is hard to improve without degrading monitorability vs. context length/”how much of the previous context is taken into account” when predicting a single token. I imagine this would be very task-dependent.