I’m trying to work out when and how ZKP (plan B) becomes more of a realistic alternative to the auditing environment (plan A), and what are the most promising areas of development to push it in the right direction. So far I have:
cost = n x C x k x p + m x c_m + f cost: total cost per year for plan A/B n: times per year to do a verification C: native compute for one forward pass k: how much more compute it takes to run an audit/verification p: price per unit of compute (only running costs, since its hardware is in f) m: number of model commitments per year c_m: cost per model commitment f: fixed costs like hardware, facility, security
Then “just” check the break-even point (I know ZKP is not really a realistic option today, but this is for how the calculation would be made in principle). If verifications per year is relatively small, then the fixed cost in plan A would presumably be the dominating contributor there, but in any case I’m trying to build a model to compare the two. There might also of course be arguments to be made to use plan B even if it costs a bit more because of the trust factor included in plan A. Now for the more interesting calculation—how to calculate k for ZKP? So far I have this:
k_ZKP = k_0 x k_h x k_m x k_t x k_b x k_a x k_s x k_p k_ZKP: how much more compute it takes to run a ZKP (than one forward pass) k_0: measured baseline* k_h: how much new hardware speeds up inference vs. ZKP** k_m: model size above measured baseline k_t: token length beyond measured baseline k_b: modification based on requirement for bit-exact verification on floating-point calculations k_a: difference in model architecture k_s: stronger requirement for security to verify in high-stakes environment k_p: progress to the ZKP function itself
*I measured a range of models up to 13B (same models, token size and GPU as the paper) and got ~2,000 − 4,000 X, roughly constant from 350M to 13B parameters (125M excluded, where the GPU is too underutilized for the ratio to be meaningful). ZKP numbers from Sun, Li, Zhang (2024) “zkLLM: Zero Knowledge Proofs for Large Language Models” https://arxiv.org/pdf/2404.16109 (Note zkLLM proves a fixed-point approximation, not bit-exact floats)
**how much newer hardware speeds up native inference relative to proving. I assume it’ll be over 1, since FP8/FP4 tensor cores help native inference but not field arithmetic. ZK-specific hardware is the counterweight.
I would really appreciate input on the calculations, is there something missing from them? As I’m new to the field of compute verification, I also want to know if this is even a worthwhile endeavor (might still do it, cause I find it interesting and as a way to learn more about the field, but wanna know if it’s worth writing up a post about it).
I’m trying to work out when and how ZKP (plan B) becomes more of a realistic alternative to the auditing environment (plan A), and what are the most promising areas of development to push it in the right direction. So far I have:
cost = n x C x k x p + m x c_m + f
cost: total cost per year for plan A/B
n: times per year to do a verification
C: native compute for one forward pass
k: how much more compute it takes to run an audit/verification
p: price per unit of compute (only running costs, since its hardware is in f)
m: number of model commitments per year
c_m: cost per model commitment
f: fixed costs like hardware, facility, security
Then “just” check the break-even point (I know ZKP is not really a realistic option today, but this is for how the calculation would be made in principle). If verifications per year is relatively small, then the fixed cost in plan A would presumably be the dominating contributor there, but in any case I’m trying to build a model to compare the two. There might also of course be arguments to be made to use plan B even if it costs a bit more because of the trust factor included in plan A.
Now for the more interesting calculation—how to calculate k for ZKP? So far I have this:
k_ZKP = k_0 x k_h x k_m x k_t x k_b x k_a x k_s x k_p
k_ZKP: how much more compute it takes to run a ZKP (than one forward pass)
k_0: measured baseline*
k_h: how much new hardware speeds up inference vs. ZKP**
k_m: model size above measured baseline
k_t: token length beyond measured baseline
k_b: modification based on requirement for bit-exact verification on floating-point calculations
k_a: difference in model architecture
k_s: stronger requirement for security to verify in high-stakes environment
k_p: progress to the ZKP function itself
*I measured a range of models up to 13B (same models, token size and GPU as the paper) and got ~2,000 − 4,000 X, roughly constant from 350M to 13B parameters (125M excluded, where the GPU is too underutilized for the ratio to be meaningful). ZKP numbers from Sun, Li, Zhang (2024) “zkLLM: Zero Knowledge Proofs for Large Language Models” https://arxiv.org/pdf/2404.16109
(Note zkLLM proves a fixed-point approximation, not bit-exact floats)
**how much newer hardware speeds up native inference relative to proving. I assume it’ll be over 1, since FP8/FP4 tensor cores help native inference but not field arithmetic. ZK-specific hardware is the counterweight.
I would really appreciate input on the calculations, is there something missing from them? As I’m new to the field of compute verification, I also want to know if this is even a worthwhile endeavor (might still do it, cause I find it interesting and as a way to learn more about the field, but wanna know if it’s worth writing up a post about it).