And most of those tricks seem unlikely to generalize beyond ARC.
Though the refinement process does sound a bit like the feared “neuralese.” I’m not too worried about this though—the problem with this kind of recurrence is that it doesn’t scale, and HRMs are indeed small models that lag SOTA. So, I don’t see much reason to expect it to work this time??
And most of those tricks seem unlikely to generalize beyond ARC.
Though the refinement process does sound a bit like the feared “neuralese.” I’m not too worried about this though—the problem with this kind of recurrence is that it doesn’t scale, and HRMs are indeed small models that lag SOTA. So, I don’t see much reason to expect it to work this time??