Thanks for your comment! Conveniently, we wrote this post about why we pick the training compute thresholds we did in the agreement (1e24 is the max). I expect you will find it interesting, as it responds to some of what you’re saying! The difference between 1e24 and 5e26 largely explains our difference in conclusions about what a reasonable unmonitored cluster size should be, I think.
You’re asking the right question here, and it’s one we discuss in the report a little (e.g., p. 28, 54). One small note on the math is that I think it’s probably better to use FP8 (so 2e15 theoretical FLOP per H100 due to the emergence of FP8 training).
Thanks for your comment! Conveniently, we wrote this post about why we pick the training compute thresholds we did in the agreement (1e24 is the max). I expect you will find it interesting, as it responds to some of what you’re saying! The difference between 1e24 and 5e26 largely explains our difference in conclusions about what a reasonable unmonitored cluster size should be, I think.
You’re asking the right question here, and it’s one we discuss in the report a little (e.g., p. 28, 54). One small note on the math is that I think it’s probably better to use FP8 (so 2e15 theoretical FLOP per H100 due to the emergence of FP8 training).