Switching to hardware keys for authentication, closing public ports on my personal server and accessing it exclusively over Tailscale, and still need to call my phone carrier to have a port-protection PIN created on my phone number. Already have a local LLM for security debugging.
I think pretty standard overall.
Perhaps
Karma: 232
If it’s not very feasible for the populace to overthrow a dictatorship, why would you expect people in power to let themselves be mind-read? They would probably say that they have national security secrets, or corporate IP secrets, which can’t be allowed to spread. They would not trust that a public provided device would not steal their secrets, and if they used a device they tuned themselves, they could manufacture agreement without actually being properly mind-read.
Especially since the real source of alignment for positions of power will come from others currently in power. It’s very unlikely that the populace will be able to decide on what values are important for a politician or public servant, they will likely be codified by existing institutions, with an eye for obedience and loyalty.
Whereas the overall populace will have no real ability to stop the government from slowly enforcing mind-reading for civil service or military positions, or positions at high-level research labs.
As for AI, I could see it being useful, but it seems like it could also prompt more concentration of power and result in an AGI aligned to a small group of people.
AI persuasion is also dangerous because an agent can easily find all information about you online, and tailor its approach to convincing you. With an evolved harness, an agent might also be very good at teasing out personality indicators from a conversation or pointed questions, and then using that to put you off-balance. It doesn’t need to convince you that it’s correct, it can just make you uncertain or distrust the sources you use for information.
This could look like bots in a forum that argue with you, in the background analyzing your account and responses to profile you, and then providing strong counterexamples to unrelated beliefs that put you off-balance.
Future of Life Foundation is doing stuff related to improving epistemics. I think surplus.dev is also funding related projects. There’s also the Foresight Institute, which did stuff on AI wisdom.
Lately I’ve been most concerned about coordination for governance, and having the ability to nicely merge values from lots of people to determine what policies a democratic government should take. I think current methods of voting leave a lot to be desired, and it should be possible to use AI to help people build personal models of their preferences that can be diffed/merged with other preferences, enabling easy creation of coalitions or identifying points of agreement. I think Andrey Tang’s post on Plurality is pretty useful for getting an idea of how similar things could work.
There are a lot of other dimensions for coordination though, getting accurate information from distorted sources, figuring out how to align representatives, better voting, etc. Mechanism design and social choice theory are good places to look for academic research on this stuff.
Even supposing that your conjecture is true, it’s not a dichotomy, we could be jealous of AI and we could be in danger of losing our jobs to it and dying to superintelligence.
I don’t believe that there is anything really there to be jealous of though. I would say for me, jealousy requires personhood, and I don’t view current LLM agents as people, so there’s nothing to envy. Moreover, even if you take as an assumption that agents are people, they definitely don’t have meaningful personalities outside of what we instill in them, and are incapable of online learning or some other way of developing independent thought. As a result, even with soul documents and RL for personas, they remain fairly average in all the ways I would want to communicate with a person, even if they excel at many tasks.
The only thing I might be jealous of an agent about is that they can clone themselves ~infinitely and run in parallel.
As for not believing humans are the best species, well, I have no doubt more optimal lifeforms could be exist, but I don’t see a point in instantiating something like that just because humanity has a bad rap sheet. I’m still having fun living, and I think lots of people are too. Maybe after everybody gets bored of regular life we can look into setting up successors.
Understand is a puzzle game that is basically what you describe as an Epistemic Roguelike, in that you deduce a different ruleset for every set of levels. It’s not an actual roguelike though, purely focused on the puzzle of figuring out rules, which are limited to a simple grid with shapes.
I would say that as something exploring a relatively unplumbed space of content, Epistemic Roguelikes are more likely to be interesting in a time where AI can make average copies of any existing content, and mainly struggles with new concepts.
Correspondingly, I think that D&D Sci might be less useful now that you could plausibly automate a large chunk of the process of creating scenarios? That’s my impression based on checking out the posts, although I haven’t actually completed one.
Yeah I would say AI is more like the popularization of PhotoShop in photography, or CGI in movies/animations. Most any work you can do with AI is a result of taking existing pieces from the medium, training the AI on them, and then using it to generate more. So it’s effectively a stochastic editor or cognitive harness, that helps you make the art. It shines in a few areas, like those subliminal pictures that are two things at once, but ultimately is like a really large sized brush that makes everything you paint average.
Wow this was a very well-written post, really put the fear of confounders in me as a humble ML student.
I took the survey
I think Canada still has a pretty outsized impact on AI, especially considering how many researchers and companies came out of places like University of Toronto or UWaterloo.
Additionally, I think that if we at least had an example of what good AI policy looks like, implemented successfully in a country, it could be used as a proof to other countries that good AI regulation is possible and useful. In that regard I think lobbying for well thought out AI regulation policies on the national level is really important, and it’s not necessary to focus on international to get traction.
I recently worked at the Canadian Department of National Defence’s newly established AI Centre, and I was surprised one of the tasks of the policy team was an AI safety report, in addition to other generic governance and responsible AI stuff. Admittedly, the AI safety report mostly focused on cybersecurity and CBRN risks, but there were some loss-of-control issues mentioned as well. It indicated to me that working to integrate AI safety by working in a department and pushing projects, or talking to director-level people is possible.
Guesses:
Bounty, magical thinking, bummer, clear division/separation, +1, made me angry, long thoughts/time spent thinking, sage, exposed an error, good communication, good post smell?/vibes, broken argument, strong argument, moved my position on the topic
There are some models on HuggingFace that do automatic PII data redaction, I’ve been working on a project to automate redaction for documents with them. AI4privacy’s models and Microsoft Presidio have been helpful.
You might find some puzzle games to be useful. In particular Understand is a game that was talked about on here as being good for learning how to test hypotheses and empirically deduce patterns. Similar to your Baba Is You experiments.
Any way that we can easily get back our own results from the survey? I know you can sometimes get a copy of your responses when you submit a Google form.
What happens to the general Lightcone portfolio if you don’t meet a fundraising target, either this year or a future year?
For concreteness, say you miss the $1M target by $200K.
The karma buttons are too small for actions that in my experience, are done a lot more than clicking to listen to the post. It’s pretty easy to misclick.
Additionally, it’s unclear what the tags are, as they’re no longer right beside the post to indicate their relevance.
I think this post would benefit from an abstract / summary / general conclusion that summarizes the main points and makes it easier to interact with. Usually I read a summary to get an idea of a post, then browse the main points and see if I’m interested enough to read on. Here, it’s hard to engage, because the writing is long and the questions it seems to deal with are nebulous.
How did you find LessWrong?
Do still have any Mormon friends? Do you want to help them break away, do you think it’s something they should do on their own, or do you find whether they remain Mormon or not immaterial?
Do you think being a Mormon was not suited for you, or do you think it doesn’t work as a way of life in general? How do you think that your answer would change 50 years ago vs today?
Did you have contact/ongoing relationships with other Mormon communities while you were there? What is the variation between people/communities? How devout/lax are different people and different communities?
How much access to the internet and the wider world did you have growing up? Were local/state/international events routinely brought up in small talk?
Thank you for writing this, and posting it here.