Proof-of-stake and proof-of-work are both often implemented cryptographically, because in cryptographic domains, verification can be easier than generation. I think another option is to apply that principle to the problem directly, where possible. The best example: formalizing a math theorem in lean means it’s much easier to verify than it is to read. CS and ML papers can sometimes (if making software is now much easier) be implemented into toy examples, sized appropriately for reviewers (and reviewers’ AI instances) to check for hardcoding or cheating. Tough data analysis domains could be turned into raw data from a reputable source plus a minimal, non-steering prompt for reviewers’ models to re-discover what the author wanted to publish. Eventually, I think automated or simulated biology/chemistry labs could be funded to attempt to reproduce new papers’ results, putting their reputations on the line.
This is not sufficient to protect against motivated bullshitters, especially not in all domains, and I think in the near future institutions may be forced to fall back more to reputation. But I think it’s workable. Academic proof-of-work is only, at best, a proxy to avoid being overloaded with verification work, but I think we can make verification easier in other ways.
Proof-of-stake and proof-of-work are both often implemented cryptographically, because in cryptographic domains, verification can be easier than generation. I think another option is to apply that principle to the problem directly, where possible. The best example: formalizing a math theorem in lean means it’s much easier to verify than it is to read. CS and ML papers can sometimes (if making software is now much easier) be implemented into toy examples, sized appropriately for reviewers (and reviewers’ AI instances) to check for hardcoding or cheating. Tough data analysis domains could be turned into raw data from a reputable source plus a minimal, non-steering prompt for reviewers’ models to re-discover what the author wanted to publish. Eventually, I think automated or simulated biology/chemistry labs could be funded to attempt to reproduce new papers’ results, putting their reputations on the line.
This is not sufficient to protect against motivated bullshitters, especially not in all domains, and I think in the near future institutions may be forced to fall back more to reputation. But I think it’s workable. Academic proof-of-work is only, at best, a proxy to avoid being overloaded with verification work, but I think we can make verification easier in other ways.