A concrete research direction in the “Searching for Search” field is to find out whether ChessGPTs or the Leela chess 0 network are searching. Your “Babble and prune” description seems checkable: maybe something like linear probe for continuation board states in the residual stream, and see if bad continuations are easier to find in earlier layers? Thank you for this writeup.
A concrete research direction in the “Searching for Search” field is to find out whether ChessGPTs or the Leela chess 0 network are searching. Your “Babble and prune” description seems checkable: maybe something like linear probe for continuation board states in the residual stream, and see if bad continuations are easier to find in earlier layers? Thank you for this writeup.