I’m talking about the precise definition of whistleblowing, not an AI that erodes privacy.
Whistleblowing is the act of reporting illegal, unethical, or unsafe activities within an organization to authorities, regulators, or the media.
And the general public seems to overwhelmingly support whistleblowers, or at least protections for whistleblowers, so I can very easily imagine them feeling the same about whistleblowing AI.
Fair enough, and I’m also grateful for those advances. I would just like to push towards good things without explicitly claiming the “playing god” mantle, which seems inherently arrogant to me; I would feel much better about someone who is trying to cure cancer if they didn’t say things like “playing god is good if you are good at it” and instead said things like “I want to advance human knowledge and try to make our lives better by defeating this disease.”
yes, playing god is good if you are good at it.
Upvoted for the honesty.
Disagree voted because, come on, everyone who plays god says the exact same thing. The whole point is that we are not gods and won’t be able to understand the consequences of our actions before they play out; in fact, this very assertion underpins the entire project of AI safety!
I wonder about this, actually. I assume all of us here would prefer that model, no? Maybe the general public feels similarly. (It seems safe to assume that CEOs/corporations would not prefer that model, though.)
Does the Fermi Paradox put an upper limit on the bounds of ASI capabilities?
Given that the Universe has had ~14 billion years to develop, it seems overwhelmingly likely that someone else out there has already maxxed out the tech tree and pushed AI as far as it can go. But we don’t see any Von Neumann probes eating the Milky Way, nor do we see any evidence of interstellar travel within the Virgo Supercluster (~147 million light years)...let alone the rest of the observable universe!
From this observation, we can conclude one of two things. Either:
FTL interstellar travel is impossible, or
We would be the first civilization to ever do it.
I concede that 2. is possible but I (perhaps naively) think 1. is far more likely. I would even go so far as to say that normal / sub-FTL interstellar travel is probably impossible on this view.
Of course, there are always the other standard possibilities e.g.
Some other civilization has achieved ASI and/or FTL, and is leaving us alone for some reason.
Maybe they’re under a Star Trek-like Prime Directive, or maybe their reasons are beyond our comprehension.
Civilizations are inherently fragile/vulnerable and always die out before reaching this level of capability.
Note that this would imply that they died for non-ASI reasons (or at least that their ASI is not infinitely power-seeking).
This possibility is also not mutually exclusive with 2.
A sub-FTL ASI/alien civilization is on its way right now and we simply haven’t been able to observe them yet.
Either way, I think we shouldn’t take the fact that we haven’t been consumed or conquered by some alien civilization or ASI (yet) for granted.
What exactly do you mean by world models, and why do you think they’re necessary? I have some initial thoughts on what I think you mean (e.g. I can imagine a future in which an AI-assisted coder or AI coder develops a program that can model external environments, which can then be used by LLMs that themselves don’t have world models) but I want to make sure we’re talking about the same thing first.
“of course AI is racist, but here’s what they don’t want you to know: they don’t know how to make it NOT be racist even if they wanted to. They literally can’t control how this thing behaves and they want it in every computer system in the planet.”
Strong upvoted because of this bit. It’s astounding that we haven’t been messaging like this the whole time; clearly people already understand “alignment” in a basic sense, and it seems straightforwardly good to work with them here instead of saying “um actually my concern is more important.”
Strong agree. Working on unconventional topics naturally places you in an epistemic bubble, which is probably fine most of the time but pretty bad if you’re working on politics/advocacy; I personally shy away from talking about AI with my non-AI-pilled friends just because I expect them to react weirdly, but to be honest, that’s probably why I should go out of my way to do it more.
Strong upvoted. “Should you reverse any advice you hear” is a problem that only applies to underspecified advice IMO, and in general I think presenting advice with more context is always a good thing.
Many would reply that these restrictions on dissemination of knowledge will drastically slow down AI research. Yes: the drastic slowing constitutes most of the benefit of imposing the restrictions.
Plan A largely doesn’t agree with this. Since they expect the slowdown treaty to eventually fall apart—at which point we revert to the status quo arms race, which we all agree is bad—they argue for slowing only insofar as it allows alignment research to outpace general AI research.
Scott Alexander had a nice little graphic on this point in his ACX post on Plan A:
Plan A also seems to largely discount the possibility of smaller research labs discovering some new paradigm that is capable of becoming an ASI without massive amounts of compute. I largely agree with this view, since ~90% of algorithmic progress is scale/compute-dependent, but either way this seems like an important crux.
I mean I agree too, I just don’t think he’d ever become a card-carrying rat (which tbf I am also not myself)
Sam Kriss is notably anti-rat so probably not
I also wonder how much context plays into this. In normal life, I usually don’t assume that my conversation partner knows what I’m talking about, so I try to give as much context as I think necessary for them to understand. On the internet you can sometimes just write for people who already know what you’re talking about.
For example I’ve noted before that I suspect current levels of AI persuasion/manipulation capabilities has lagged behind AI capabilities overall (this is nonobvious and I don’t defend it here but my impression from talking to empirical researchers on AI persuasion is that they broadly agree with me).
IMO this is also visible in AI writing. I would imagine persuasion and writing capabilities should go hand in hand, and while AI is getting better at producing stuff people are willing to read, it’s still obviously not a human-replacement level (and indeed has mostly succeeded in automating formulaic/procedural writing like summaries and lit reviews). Will be interesting to see this going forward.
I have a suspicion that p-zombie discourse is only going to get more relevant as LLMs get better. No one really argues that animals aren’t conscious, even though they can’t use words very well, but the release of GPT-3 caused a steady rise in people arguing that AIs are conscious. It’s not clear to me that an LLM couldn’t possibly be conscious, but it does seem that many people are taking LLM eloquence to imply that they are conscious, and I’m pretty sure we’ve been discussing this for years…
Ooh very interesting. I fall for throat-clearing a lot myself. I do think it’s kinda hard to avoid context and overview, though, and intros can be nice stylistically? But maybe this is just me making excuses for my bad habits...
Some conjectures:
General cultural ossification: caused by widespread recording and information technology devices.
Cultural splintering: things change so fast nowadays that there simply isn’t time for new symbols to catch on before they’re replaced by even newer symbols (and culture is also so fragmented that there isn’t really a single power center that can take over).
Standardization: now that symbols tend to be standard across international communities, changing things might require a LOT of coordination.
Low hanging fruit has already been plucked: potential improvements (if they exist at all) might be minor and not worth overhauling already-standardized systems.
Symbols are still developing: how long has the like button been around? Or the karma system? Maybe we’ll keep innovating our symbology as new needs arise.
I wish he’d given some common failure modes and ways to fix them. Like I completely agree with the main point, but without concrete examples, I have a hard time applying this advice to my own writing except for “try harder dingus” which is often unhelpful.
A related example from my own life:
Bob: “You should be wary of Charlie.“
Alice: “Why? He’s only ever been nice to me!”
Bob: “Yes, and he’s also only ever been nice to me, but many of our friends report him being bad to them AND being horrible to people who he seems to consider ‘beneath him.’ I trust our friends, and so I’m worried about the possibility that Charlie is specifically kind when he’s around us.”
Alice: “Charlie isn’t the type of person who would do that. I can tell.”
Bob: “…”