But there are systems that work better with lower bandwidth or have deliberately lower bandwidth, like autoencoders.
I understand that the bandwidth is certainly higher for one than the other, but this both might not be an advantage in this circumstance or could be an advantage in some respects but a greater disadvantage in others.
The point of an autoencoder is to form good representations, not to perform well. I’m struggling to think of any other examples where low bandwidth is good, that arent just implementation issues (and, again, in current systems text CoT > neuralese, so obviously low bandwidth can be good)
But there are systems that work better with lower bandwidth or have deliberately lower bandwidth, like autoencoders.
I understand that the bandwidth is certainly higher for one than the other, but this both might not be an advantage in this circumstance or could be an advantage in some respects but a greater disadvantage in others.
The point of an autoencoder is to form good representations, not to perform well. I’m struggling to think of any other examples where low bandwidth is good, that arent just implementation issues (and, again, in current systems text CoT > neuralese, so obviously low bandwidth can be good)