Regarding algorithmic similarity. This is an idea I just thought of this moment, so I’m not sure how solid it is, but. Given Turing machines M1 and M2 that compute the same functions, we want to say whether in some sense they do it “in the same way”. Let’s consider, for any input x, the entire histories of intermediate states of the computations M1(x) and M2(x). Call them h1(x) and h2(x). We then say that M1 and M2 are “algorithmically equivalent” when there is a low complexity algorithm A that, given access to h1(x), can produce any given part of h2(x), and vice versa. In particular, the complexity of A must be much lower than the complexity of running Mi(x) from the beginning. Here, it seems useful to play with different types of complexity bounds (including time/space for example).
Regarding waterfalls and human beings. I think that a waterfall is not simulating a human being, because there is no algorithm of simultaneously low description complexity and low computational complexity that can decode a human being from a waterfall. Ofc it is not a binary distinction but a fuzzy distinction (the simpler the decoding algorithm is, the more reasonable it is to say a human being is there).
Regarding diamond optimizers. I think that the right way to design such an optimizer would be using an instrumental reward function. We then remain with the problem of how to specify this function. We could start with some ontology or class of ontologies that can be reasonably said to contain diamonds, and for which we can define the reward function unambiguously. These ontologies are then mapped into the space of instrumental states, giving us a partial specification of the instrumental reward function (it is specified on the affine span of the images of the ontologies). Then, there is the quesiton of how to extend the reward function to the entire instrumental state space. I wrote a few thoughts about that in the linked essay, but another approach we can take is, considering all extensions that have same range of values. These form a convex set, that can be interpreted as Knightian uncertainty regarding the reward function. We can then consider maximin policies for this set to be “diamond maximizers”. In other words, we want the maximizer to be cautious/conservative about judging the number of diamonds on states that lie outside the ontologies.
I definitely think the computational complexity approach is worth looking into, though I think computational complexity behaves kind of weirdly at low complexities.
I like the view that waterfalls are at least a bit conscious! Definitely goes against my own intuition.
I’m a bit worried that whether or not there is a low description complexity and low computational complexity algorithm that decodes a human from a waterfall might depend heavily on how we encode the waterfall as a mathematical object and that although it would be clear for “natural” encodings that it was unlike a human we might need a theory to tell us which encodings are natural and which are not.
Not sure what do you mean by “computational complexity behaves kind of weirdly at low complexities”? In this case, I would be tempted to try the complexity class L (logarithmic space complexity).
The most natural encoding is your “qualia”, your raw sense data. This still leaves some freedom for how do you represent it, but this freedom has only a very minor effect.
A few comments:
Regarding algorithmic similarity. This is an idea I just thought of this moment, so I’m not sure how solid it is, but. Given Turing machines M1 and M2 that compute the same functions, we want to say whether in some sense they do it “in the same way”. Let’s consider, for any input x, the entire histories of intermediate states of the computations M1(x) and M2(x). Call them h1(x) and h2(x). We then say that M1 and M2 are “algorithmically equivalent” when there is a low complexity algorithm A that, given access to h1(x), can produce any given part of h2(x), and vice versa. In particular, the complexity of A must be much lower than the complexity of running Mi(x) from the beginning. Here, it seems useful to play with different types of complexity bounds (including time/space for example).
Regarding waterfalls and human beings. I think that a waterfall is not simulating a human being, because there is no algorithm of simultaneously low description complexity and low computational complexity that can decode a human being from a waterfall. Ofc it is not a binary distinction but a fuzzy distinction (the simpler the decoding algorithm is, the more reasonable it is to say a human being is there).
Regarding diamond optimizers. I think that the right way to design such an optimizer would be using an instrumental reward function. We then remain with the problem of how to specify this function. We could start with some ontology or class of ontologies that can be reasonably said to contain diamonds, and for which we can define the reward function unambiguously. These ontologies are then mapped into the space of instrumental states, giving us a partial specification of the instrumental reward function (it is specified on the affine span of the images of the ontologies). Then, there is the quesiton of how to extend the reward function to the entire instrumental state space. I wrote a few thoughts about that in the linked essay, but another approach we can take is, considering all extensions that have same range of values. These form a convex set, that can be interpreted as Knightian uncertainty regarding the reward function. We can then consider maximin policies for this set to be “diamond maximizers”. In other words, we want the maximizer to be cautious/conservative about judging the number of diamonds on states that lie outside the ontologies.
I definitely think the computational complexity approach is worth looking into, though I think computational complexity behaves kind of weirdly at low complexities.
I like the view that waterfalls are at least a bit conscious! Definitely goes against my own intuition.
I’m a bit worried that whether or not there is a low description complexity and low computational complexity algorithm that decodes a human from a waterfall might depend heavily on how we encode the waterfall as a mathematical object and that although it would be clear for “natural” encodings that it was unlike a human we might need a theory to tell us which encodings are natural and which are not.
Not sure what do you mean by “computational complexity behaves kind of weirdly at low complexities”? In this case, I would be tempted to try the complexity class L (logarithmic space complexity).
The most natural encoding is your “qualia”, your raw sense data. This still leaves some freedom for how do you represent it, but this freedom has only a very minor effect.