Re personal opinion: what is your take on the feasibility of human experiments? It seems like your model is compatible with IDA working out even though no-one can ever demonstrate something like ‘solve the hardest exercise in a textbook’ using participants with limited time who haven’t read the book.
Yeah, that seems right to me—I don’t really expect to see us solving hard exercises in a textbook with a small number of humans without any additional tricks. I don’t think Ought did either; from pretty early on they were talking about strategies for having larger trees, e.g. via automated decomposition strategies, or caching / memoization of strategies, possibly using ML.
In addition, I think Ought historically has pursued the strategy “try the thing that, if successful, would allow us to build a safety story”, rather than “try the thing that, if it fails, implies that factored cognition would not work out”, which is why they talk about particularly challenging tasks like solving the hardest exercise in a textbook.
Re personal opinion: what is your take on the feasibility of human experiments? It seems like your model is compatible with IDA working out even though no-one can ever demonstrate something like ‘solve the hardest exercise in a textbook’ using participants with limited time who haven’t read the book.
Yeah, that seems right to me—I don’t really expect to see us solving hard exercises in a textbook with a small number of humans without any additional tricks. I don’t think Ought did either; from pretty early on they were talking about strategies for having larger trees, e.g. via automated decomposition strategies, or caching / memoization of strategies, possibly using ML.
In addition, I think Ought historically has pursued the strategy “try the thing that, if successful, would allow us to build a safety story”, rather than “try the thing that, if it fails, implies that factored cognition would not work out”, which is why they talk about particularly challenging tasks like solving the hardest exercise in a textbook.