Also, while Amazon AWS is arguably the biggest player in cloud computing generally, I have heard (though not independently vetted) that AWS is rarely used for training cutting-edge LLMs. Because compared to some other compute providers, Amazon’s compute is so geographically distributed and not centralized enough for the purpose of training very large models.
I don’t think this is the reason. Rare is the training run that’s so big it doesn’t fit comfortably in what you can buy in a single Amazon datacenter. I think the real reason is that AWS has significantly larger margins than most cloud providers, since their offering is partially a SaaS offering.
I don’t think this is the reason. Rare is the training run that’s so big it doesn’t fit comfortably in what you can buy in a single Amazon datacenter. I think the real reason is that AWS has significantly larger margins than most cloud providers, since their offering is partially a SaaS offering.