Andrew Trask together with Ben Garfinkel, Allan Dafoe and Eric Drexler (and others I know less well) released a paper on transparency and infohazards that seemed interesting on a first skim:
Abstract: Successful collaboration involves sharing information. However, parties may disagree on how the information they need to share should be used. We argue that many of these concerns reduce to ‘the copy problem’: once a bit of information is copied and shared, the sender can no longer control how the recipient uses it. From the perspective of each collaborator, this presents a dilemma that can inhibit collaboration. The copy problem is often amplified by three related problems which we term the bundling, edit, and recursive enforcement problems. We find that while the copy problem is not solvable, aspects of these amplifying problems have been addressed in a variety of disconnected fields. We observe that combining these efforts could improve the governability of information flows and thereby incentivise collaboration. We propose a five-part framework which groups these efforts into specific capabilities and offers a foundation for their integration into an overarching vision we call “structured transparency”. We conclude by surveying an array of use-cases that illustrate the structured transparency principles and their related capabilities.
Collaboration requires sharing information amongst participants. This gives rise to the central problem of information governance, which we call the copy problem: after replicating and sharing a bit of information, the sender can no longer control how the recipient might use it. The copy problem creates a challenging trade-off for would-be collaborators because each participant must make an educated guess about the likelihood and impact of others’ misuse of the shared data. Assuming rationality, each participant weighs the expected cost of this process against the expected benefit of the collaboration. If the latter outweighs the former for all parties, then the collaboration proceeds. However, if a would-be collaborator expects costs to exceed rewards, they may limit or block the collaboration. Relevant risks and costs informing this assessment can include privacy, security, legal, IP, competitive, public relations, and other similar considerations.
The first contribution of this paper is to lay out the copy problem, which poses serious issues because it cannot presently be solved (information cannot be controlled once shared) and because it scales quadratically in the size of a collaboration (n parties must satisfy n(n-1) data-sharing relationships to proceed)1 . Since this dilemma applies anywhere information would be shared in a collaboration, it has a broad impact.
The second contribution of this paper is to argue that three additional issues, which we term the bundling, edit, and recursive enforcement problems, exacerbate the copy problem. In many cases, these challenges can be solved, reducing the impact of the copy problem. However, we observe that progress on these issues is fragmented across many disciplines, such as machine learning, cryptography, distributed systems, database theory, statistics, probability, political science, and legal studies.
As the third contribution, this work re-frames these related-but-disconnected approaches as a shared aspiration for structured transparency (e.g., the ability for actors to reduce collaboration risks and costs by defining and enforcing precise flows of information) and organises techniques under a five-part framework. This helps point out which approaches offer identical capabilities and which are complementary to one another. We hope that this framework will serve as a foundation for integrating solutions to the edit, bundling, and recursive enforcement problems, and, in turn, dull the effect of the copy problem and decrease the informational cost of collaborations.
The fourth contribution of this work describes technical approaches for structured transparency, especially a set of techniques known as privacy enhancing technologies (PETs). We find that, when viewed through the lens of structured transparency, PETs offer enhancements far beyond that of privacy, addressing the edit, bundling, and recursive enforcement problems more generally — with relevance for many of the factors that weigh on collaboration decisions (security, legal, IP, competition, public relations, etc.). Finally, we illustrate the contributions of PETs within the structured transparency principles via a set of real-world use cases.
Structured Transparency: a framework for addressing use/mis-use trade-offs when sharing information
Link post
Andrew Trask together with Ben Garfinkel, Allan Dafoe and Eric Drexler (and others I know less well) released a paper on transparency and infohazards that seemed interesting on a first skim:
Rest of the paper: https://arxiv.org/pdf/2012.08347.pdf