This is an interesting theory (one I would love to read more about) but this neglects that the vast majority of people in Rome would not have had access to many of these energy sources. About 75-90% of people lived in rural areas, never seeing the famous baths or mud-bricks. Many who would go into the army came from these areas, only every seeing the glory of Rome itself in a preceding triumph. But I think this theory might hold for energy efficiency in agriculture.
Rome went through many different systems throughout its history: first the small-scale farmers, then the Latifundia and slave-estates, then the local patronage system (circa 300AD), and then fully developed manor estates (sort of like medieval feudalism). One could trace the rise and fall of Rome with the changing of these different agricultural systems, so I wonder if later systems scarified efficiency for stability.
On the other hand, I shouldn’t mistake correlation for causation, and so it could be that these systems didn’t cause the rise and fall of Rome, but were merely a reaction to it.
It would really depend on how many parameters the model has IMO, if the jump from GPT-3 to GPT-4 is something on the order of magnitude of 10-100x, then we could potentially see similar gains for multiplication. GPT-3 (175B) can do 2 digit multiplication with a ~50% accuracy, so 5-6 digits might be possible. It really depends on how well the model architecture of GPT scales in the future.