Can a DL-based system still end up causing catastrophic damage before we ever even manage to get to ASI?
AGISF 2022 Summaries
Hey, I just wanted to write a quick update. Since you mentioned you will be using the 2023 summaries around Feb. Unfortunately, it seems like the AGISF syllabus is still very fluid, and the readings are still changing as the current iteration of the course is progressing. Which basically means, that the only realistic target for getting those done, is by the end of this current AGISF iteration. Sorry if that causes inconvenience.
Is AI Safety dropping the ball on privacy?
Thanks for pointing that out! It’s embarrassing that I made a mistake, but it’s also relieving in some sense to learn that the impacts were not as I had thought them to be.
I hope this error doesn’t serve to invalidate the entire post. I don’t really know what the post-publishing editing etiquette is, but I don’t want to keep anything in the post that might serve as misinformation so I’ll edit this line out.
Please let me know if there are any other flaws you find and I’ll get them fixed.
I did consider the distinction between a model of humans vs. a model of you personally. But I can’t really see any realistic way of stopping the models from having better models of humans in general over time. So yeah, I agree with you that the small pockets of sanity are currently the best we can hope for. It was mainly to spread the pocket of sanity from infosec to the alignment space is why I wrote up this post. Because I would consider the minds of alignment researchers to be critical assets.
As to why predictive models of humans in general seems unstoppable—I thought it might be too much to ask to not even provide anonymized data because there are a lot of good capabilities that are enabled by that (e.g. better medical diagnoses). Even if it is not too heavy of a capability loss most people would still provide data because they simply don’t care or remain unaware. Which is why I used the wording—stem the flow of data and delay timelines instead of stopping the flow.
I am trying to be as realistic as I can while realizing that privacy is inversely proportional to convenience.
So no, of course you should not stop making lesswrong posts.
The main things I suggested were—removing the ability to use data by favoring E2EE, and additionally removing the ability to hoard data, by favoring decentralized (or local) storage and computation.
As an example just favor E2EE services for collaborating instead of drive, dropbox, or office suite if you have the ability to do so. I agree that this doesn’t solve the problem but at least it gets people accustomed to thinking about using privacy-focused alternatives. So it is one step.
Another example would be using an OS which has no telemetry and gives you root access, both on your computer and on your smartphone.
There is a different class of suggestions that fall under digital hygiene in general, but as mentioned in the - ‘what is this post about’ section, that is not what this post is about. I am also intentionally avoiding naming the alternative services because I didn’t want this post to come across as a shill.
Also, this is all a question of timelines. If people think that AGI/ASI rears its head within the next years or decade, I would agree that there might be bigger fires to put out.
I understand your original comment a lot better now. My understanding of what you said is that open source intelligence that anyone provides through their public persona is revealing more than enough information to be damaging. The little that is sent over encrypted channels is just cherries on the cake. So the only real way to avoid manipulation is to first hope that you have not been a very engaged member of the internet for the last decade, and also primarily communicate over private channels.
I suppose I just underestimated how much people actually post stuff online publicly.
One first instinct response I had was identity isolation. That was something I was going to suggest while writing the original post as well. Practicing identity isolation would mean that even if you post anything publicly the data is just isolated to that identity. Every website, every app, is either compartmentalized or is on a completely different identity. Honestly, though that requires almost perfect OPSEC to not be fingerprintable. Besides just that, it’s also way too inconvenient for people to not just use the same email, and phone number or just log in with google everywhere. So even though I would like to suggest it, no one would actually do it. And as you already mentioned most normal people have just been providing boatloads of free OSINT for the last few decades anyway...
Thinking even more, the training dataset over the entire public internet is basically the centralized database of your data that I am worried about anyway. As you mentioned AIs can find feature representations that we as humans cant. So basically even if you have been doing identity isolation, LLMs (or whatever future model) would still be able to fingerprint you as long as you have been posting enough stuff online. And not posting online is not something that most people are willing to do. Even if they are, they have already given away the game by what they have already posted. So in a majority of cases, identity isolation doesn’t help this particular problem of AI manipulation either...
I have always tried to hold the position that even if it might be possible for other people (or AIs) to do something you don’t like (violate privacy/manipulate you), that doesn’t mean you should give up or that you have to make it easy for them. But off the top of my head, without thinking about this some more I can’t really come up with any good solution for people who have been publicly publishing info on the internet for a while. Thank you for giving me food for thought.
Hey Dusan! Yes, Of course, you have permission to translate these summaries. It’s awesome that you are doing that!
Thanks for your suggestion. Yeah, this comment serves as blanket permission to anyone who wants to translate to freely do so.