I think the way you expressed issue 3 makes it too much of a clone of issue 1; if I tell you the bounds for the question in terms of programs, then I think there is a general way to apply SI to get a sensible bounded answer. If I tell you the bounds in terms of functions, then there would be a general way to incorporate that info into SI, if you knew how to move between functions and programs.
The way I think about those issues that (I think?) separates them more cleanly is that we both have to figure out the ‘compression’ problem of how to consider ‘models’ as families of programs (at some level of abstraction, at least) and the ‘elaboration’ problem of how to repopulate our stable of candidates when we rule out too many of the existing ones. SI bypasses the first and gives a trivial answer to the second, but a realistic intelligence will have interesting answers to both.
I agree with those issues.
I think the way you expressed issue 3 makes it too much of a clone of issue 1; if I tell you the bounds for the question in terms of programs, then I think there is a general way to apply SI to get a sensible bounded answer. If I tell you the bounds in terms of functions, then there would be a general way to incorporate that info into SI, if you knew how to move between functions and programs.
The way I think about those issues that (I think?) separates them more cleanly is that we both have to figure out the ‘compression’ problem of how to consider ‘models’ as families of programs (at some level of abstraction, at least) and the ‘elaboration’ problem of how to repopulate our stable of candidates when we rule out too many of the existing ones. SI bypasses the first and gives a trivial answer to the second, but a realistic intelligence will have interesting answers to both.