Happy to try to clarify, and this is helping me rethink my own thoughts, so appreciate the prompts. I’m playing with new trains of thought here and so have pretty low confidence in where I ended up, so greatly appreciate any further clarifications or responses you have.
There’s a standard trick for scoring an uncertain prediction: It outputs its probability estimate p
Yup, understand that is how to effectively score uncertainty. I was very wrong to phrase this as “we still have to have some framework to map uncertainty to a state” because you don’t strictly have to do anything, you can just use probabilities.
Restricting this to discrete, binary states allows us to simplify the comparison between models for this discussion. I will claim we can do so with no loss of fidelity (leaning heavily on Shannon, ie, this is all just information, encoding it to binary and back out again doesn’t mess anything up). And doing so is not obliged, but useful.
I really shouldn’t have said “you must X!” I should have said “it’s kind of handy if you X,” sorry for that confusion.
You’re saying that giving it less information (by replacing its camera feed with a lower quality feed) is equivalent to sometimes lying to it? I don’t see the equivalence!
We have a high quality information stream and a low quality information stream, and they both gesture vaguely at the ultimate high quality information stream, namely, the true facts of the matter of the world itself. Say, LQ < HQ < W.
LQ may be low quality because it is missing information in HQ, it may just be a subset of HQ, like a lower resolution video. Or it may have actual noise, false information.
If we have a powerful algorithm, we may be able to, at least asymptomatically, convert LQ to HQ, using processing power. So maybe in some cases LQ + processing = HQ exactly. But that makes the distinction uninteresting, and you would likely have to further degrade v′1 to get the effect you are looking for, so let’s discard that and consider only cases where v′1 is strictly worse.
You can now use a NAND to sort the outputs of LQ and HQ into two buckets:
A stream of outputs that all agree.
A stream of outputs that all disagree.
So for bucket 1, there are aspects of the world where there’s effectively no loss in quality. But comparing HQ with HQ is not useful, so let’s discard those cases, and examine the corners where LQ and HQ disagree.
LQ effectively has false information about some subset of reality there, that is in a sense what “LQ” means.
(Or just has gaps, which resolve to approximate HQ after processing, or fail and resolve to noise, either way.)
if you overfit on preventing human simulation, you let direct translation slip away
Rereading, I think HoldenK started down this path, “once the predictor is good enough that it can get data points right despite missing crucial information, it is also (potentially) good enough that it can learn how to imitate “what the human would think had happened if they had more information.”″
So for your block—in a sense you’re giving the human some information the predictor lacks. You’re giving the human “hints,” in the form of higher quality input, which helps get the human closer to perfectly representing the actual world. (Not completely, sometimes there’s still uncertainty, but closer than the predictor is likely to get.)
If that gets the human to “perfect”, then the best the predictor can do is asymptotically approach human prediction and direct translation at the same time.
My Weak Spots
I think one likely objection to what I wrote here is that I am abusing Shannon. I’ve considered that, would be happy to discuss it more and carefully consider objections along those lines, but I think toy examples would get us there. And without taking away from your notes about how “Sometimes the predictor’s probability is strictly between 0 and 1, so it gets some loss.” If p(I eat soup) is 0.6 for all days, let’s just ask ten discrete questions, “across n days the number of soups I eat will converge to n/1? (T/F), n/2? (T/F), …” I would definitely try to preserve performance and scoring, I just want to run the NAND.
I think another likely objection is that when we apply models, trying to get m(HQ) = ~W, then it relies on interactions of states in complex ways where we can’t slice them randomly into two groups without disrupting how models work at the basic level. I think the response is to simply group these states into bigger subsets of outcomes and treat those as atomic.
I think the biggest and most important objection would be that I’ve misunderstood your block. I would welcome any clarifications, and especially appreciate a toy example if you could, even if not involving diamonds, just to make sure I definitely get what you’re saying in that part.
I’d be interested in other objections or weak spots here, appreciate your time helping me to think this through more carefully and completely.
Happy to try to clarify, and this is helping me rethink my own thoughts, so appreciate the prompts. I’m playing with new trains of thought here and so have pretty low confidence in where I ended up, so greatly appreciate any further clarifications or responses you have.
Yup, understand that is how to effectively score uncertainty. I was very wrong to phrase this as “we still have to have some framework to map uncertainty to a state” because you don’t strictly have to do anything, you can just use probabilities.
Restricting this to discrete, binary states allows us to simplify the comparison between models for this discussion. I will claim we can do so with no loss of fidelity (leaning heavily on Shannon, ie, this is all just information, encoding it to binary and back out again doesn’t mess anything up). And doing so is not obliged, but useful.
I really shouldn’t have said “you must X!” I should have said “it’s kind of handy if you X,” sorry for that confusion.
We have a high quality information stream and a low quality information stream, and they both gesture vaguely at the ultimate high quality information stream, namely, the true facts of the matter of the world itself. Say, LQ < HQ < W.
LQ may be low quality because it is missing information in HQ, it may just be a subset of HQ, like a lower resolution video. Or it may have actual noise, false information.
If we have a powerful algorithm, we may be able to, at least asymptomatically, convert LQ to HQ, using processing power. So maybe in some cases LQ + processing = HQ exactly. But that makes the distinction uninteresting, and you would likely have to further degrade v′1 to get the effect you are looking for, so let’s discard that and consider only cases where v′1 is strictly worse.
You can now use a NAND to sort the outputs of LQ and HQ into two buckets:
A stream of outputs that all agree.
A stream of outputs that all disagree.
So for bucket 1, there are aspects of the world where there’s effectively no loss in quality. But comparing HQ with HQ is not useful, so let’s discard those cases, and examine the corners where LQ and HQ disagree.
LQ effectively has false information about some subset of reality there, that is in a sense what “LQ” means.
(Or just has gaps, which resolve to approximate HQ after processing, or fail and resolve to noise, either way.)
Rereading, I think HoldenK started down this path, “once the predictor is good enough that it can get data points right despite missing crucial information, it is also (potentially) good enough that it can learn how to imitate “what the human would think had happened if they had more information.”″
So for your block—in a sense you’re giving the human some information the predictor lacks. You’re giving the human “hints,” in the form of higher quality input, which helps get the human closer to perfectly representing the actual world. (Not completely, sometimes there’s still uncertainty, but closer than the predictor is likely to get.)
If that gets the human to “perfect”, then the best the predictor can do is asymptotically approach human prediction and direct translation at the same time.
My Weak Spots
I think one likely objection to what I wrote here is that I am abusing Shannon. I’ve considered that, would be happy to discuss it more and carefully consider objections along those lines, but I think toy examples would get us there. And without taking away from your notes about how “Sometimes the predictor’s probability is strictly between 0 and 1, so it gets some loss.” If p(I eat soup) is 0.6 for all days, let’s just ask ten discrete questions, “across n days the number of soups I eat will converge to n/1? (T/F), n/2? (T/F), …” I would definitely try to preserve performance and scoring, I just want to run the NAND.
I think another likely objection is that when we apply models, trying to get m(HQ) = ~W, then it relies on interactions of states in complex ways where we can’t slice them randomly into two groups without disrupting how models work at the basic level. I think the response is to simply group these states into bigger subsets of outcomes and treat those as atomic.
I think the biggest and most important objection would be that I’ve misunderstood your block. I would welcome any clarifications, and especially appreciate a toy example if you could, even if not involving diamonds, just to make sure I definitely get what you’re saying in that part.
I’d be interested in other objections or weak spots here, appreciate your time helping me to think this through more carefully and completely.