As someone who has worked at Arbital (though I joined after Alexei went on break, so he might think differently, and I never talked to Eliezer directly during my time there), my read is that the things that Eliezer did write down were not comprehensive enough to build a coherent model of his internal model. Multiple long conversations with Eliezer did also not seem good enough to get someone to build a strong enough model so that they wouldn’t very quickly propose things that Eliezer thought were very bad ideas. Having the notes and conversations publicly, allowing more people to study them, might help here – but I am currently skeptical that that would be enough for anyone to successfuly extract Eliezer’s models.
As someone who has worked at Arbital (though I joined after Alexei went on break, so he might think differently, and I never talked to Eliezer directly during my time there), my read is that the things that Eliezer did write down were not comprehensive enough to build a coherent model of his internal model. Multiple long conversations with Eliezer did also not seem good enough to get someone to build a strong enough model so that they wouldn’t very quickly propose things that Eliezer thought were very bad ideas. Having the notes and conversations publicly, allowing more people to study them, might help here – but I am currently skeptical that that would be enough for anyone to successfuly extract Eliezer’s models.
I see, thanks. Does anyone other than Eliezer have a good idea of what Eliezer wanted (wants?) to build?
Nate Soares had some strong opinions, and I generally expect him to have good models of Eliezer, but he is not any less busy than Eliezer is.