Yes, I think this does matter a lot. Extremely long CoTs (beyond 1 million tokens) will degrade in some manner due to compaction, which will make the graph of score vs tokens spent become shallower (and could be what forces it to plateau after millions of tokens). This will lessen the gap between the two and may eventually make swarm scaling exceed CoT scaling.
This appears in my analysis where I say:
There may also be other advantages to multiple agents on top of speed. For example, if you keep increasing the length of the chain of thought for a single agent, the performance eventually plateaus. But the height of the plateau for a 1,000-agent swarm may be greater than for a single agent.
But ideally I would have mentioned compaction as a specific mechanism for this.
Yes, I think this does matter a lot. Extremely long CoTs (beyond 1 million tokens) will degrade in some manner due to compaction, which will make the graph of score vs tokens spent become shallower (and could be what forces it to plateau after millions of tokens). This will lessen the gap between the two and may eventually make swarm scaling exceed CoT scaling.
This appears in my analysis where I say:
But ideally I would have mentioned compaction as a specific mechanism for this.