Yeah, sorry, poor wording on my part. What I meant in that part was “argue that the direct translator cannot be arbitrarily complex”, although I immediately mention the case you’re addressing here in the parenthetical right after what you quote.
Ah, I just totally misunderstood the sentence, the intended reading makes sense.
Well, it might be that a proposed solution follows relatively easily from a proposed definition of knowledge, in some cases. That’s the sort of solution I’m going after at the moment.
I agree that’s possible, and it does seem like a good reason to try to clarify a definition of knowledge.
Ahh, I see. I had 100% interpreted the computational complexity of the Reporter to be ‘relative to the predictor’ already. I’m not sure how else it could be interpreted, since the reporter is given the predictor’s state as input, or at least given some form of query access.
What’s the intended mathematical content of the statement “the direct translation can be arbitrarily complex”, then?
Sorry, what I mean is:
The computational complexity of the reporter can be arbitrarily large.
But it’s not clear the computational complexity of the reporter can be arbitrary larger than the predictor.
E.g. maybe the reporter can have complexity 0.1% of the predictor’s complexity, but that means that the reporter gets arbitrarily complex in the limit where the predictor is arbitrarily complex.
Also, why don’t you think the direct translator can be arbitrarily complex relative to the predictor?
I assume this was based on my confusing use of “relative to.” But answering just in case: if we are defining “knowledge” in terms of what the predictor actually uses in order to get a low loss, then there’s some hope that the reporter can’t really be more complex than the predictor (for the part that is actually playing a role in the predictor’s computation) plus a term that depends only on the complexity of the human’s model.
Ah, I just totally misunderstood the sentence, the intended reading makes sense.
I agree that’s possible, and it does seem like a good reason to try to clarify a definition of knowledge.
Sorry, what I mean is:
The computational complexity of the reporter can be arbitrarily large.
But it’s not clear the computational complexity of the reporter can be arbitrary larger than the predictor.
E.g. maybe the reporter can have complexity 0.1% of the predictor’s complexity, but that means that the reporter gets arbitrarily complex in the limit where the predictor is arbitrarily complex.
I assume this was based on my confusing use of “relative to.” But answering just in case: if we are defining “knowledge” in terms of what the predictor actually uses in order to get a low loss, then there’s some hope that the reporter can’t really be more complex than the predictor (for the part that is actually playing a role in the predictor’s computation) plus a term that depends only on the complexity of the human’s model.