the neural network is not built to be able to understand a goal like “((A or B) and (C or D))
Could this not be specified as “(A and C) or (A and D) or (B and C) or (B and D)” ?
I guess for a complex goal it might end up with an unfeasibly large disjunction when forced into that format. But I would still expect a lot of goals could be expressed by enumeration of all the combinations that would satisfy them.
Yes, one could express the conjunction ((A or B) and (C or D)) as the slightly longer, equivalent disjunction ((A and C) or (A and D) or (B and C) or (B and D)), but it’s worth pointing out that in addition to being longer, the disjunction is also highly structured… to wit, each individual atom appears twice in a systematic pattern. If atoms are chosen at random to fill in the slots, then you’re vanishingly unlikely to get a training goal that corresponds to a simple conjunction like the original. Of course, you could hand-code it to sometimes intentionally include examples that have the correct structure… but then, ah, you may as well code it to understand conjunctions directly, I think?
Yeah, I said that badly. It isn’t precisely the lack of expressiveness that bugs me. You’re 100% right about the equivalencies.
Instead, it’s that the grammar for OR is built into the system at a deep level; that the goal-attention module has separate copies of itself getting as input however many As, Bs, amd Cs are in “A or B or C”.
Like—it makes sense to think of the agent as receiving the goals, given how they’ve set it up. But it doesn’t make sense to think of the agent as receiving the goals in language, because language implies a greater disconnect between mental modules and words than it’s set up to have. Which again, isn’t so much a problem with the paper as it is an easy over-estimation of the paper’s accomplishments.
One small thing:
Could this not be specified as “(A and C) or (A and D) or (B and C) or (B and D)” ?
I guess for a complex goal it might end up with an unfeasibly large disjunction when forced into that format. But I would still expect a lot of goals could be expressed by enumeration of all the combinations that would satisfy them.
Yes, one could express the conjunction ((A or B) and (C or D)) as the slightly longer, equivalent disjunction ((A and C) or (A and D) or (B and C) or (B and D)), but it’s worth pointing out that in addition to being longer, the disjunction is also highly structured… to wit, each individual atom appears twice in a systematic pattern. If atoms are chosen at random to fill in the slots, then you’re vanishingly unlikely to get a training goal that corresponds to a simple conjunction like the original. Of course, you could hand-code it to sometimes intentionally include examples that have the correct structure… but then, ah, you may as well code it to understand conjunctions directly, I think?
Yeah, I said that badly. It isn’t precisely the lack of expressiveness that bugs me. You’re 100% right about the equivalencies.
Instead, it’s that the grammar for OR is built into the system at a deep level; that the goal-attention module has separate copies of itself getting as input however many As, Bs, amd Cs are in “A or B or C”.
Like—it makes sense to think of the agent as receiving the goals, given how they’ve set it up. But it doesn’t make sense to think of the agent as receiving the goals in language, because language implies a greater disconnect between mental modules and words than it’s set up to have. Which again, isn’t so much a problem with the paper as it is an easy over-estimation of the paper’s accomplishments.