I believe that given a few years, a company wanting to make a 10 trillion parameter GPT-3 could probably do it for less than these estimates, since 71-358 million usd isn’t that much in compute, and for that money extra specialized hardware produced in bulk could be used to bring costs down.
I believe that given a few years, a company wanting to make a 10 trillion parameter GPT-3 could probably do it for less than these estimates, since 71-358 million usd isn’t that much in compute, and for that money extra specialized hardware produced in bulk could be used to bring costs down.