I don’t see how this breaks CoT. The memory module in Titans stores surprising information as it’s encountered and then allows the transformer to look at it later on, but it doesn’t synthesize new information. Strikes me as two entirely compatible augmentations of the transformer architecture.
I don’t see how this breaks CoT. The memory module in Titans stores surprising information as it’s encountered and then allows the transformer to look at it later on, but it doesn’t synthesize new information. Strikes me as two entirely compatible augmentations of the transformer architecture.
This seems right—I was confused about the original paper. My bad.