I am Andrew Hyer, currently living in New Jersey and working in New York (in the finance industry).
aphyer
Karma: 5,672
Does this make LLMs much more deterministic? Suppose I ask an LLM to generate a D&D character for me, and do this several hundred times to make a bunch of random NPC adventurers.
An unmodified LLM might make some random decisions and produce a hundred different D&D characters.
A watermarked LLM will watermark itself by reliably producing a half-elf wizard named Erastus, and will produce the same character one hundred times. This will, indeed, let you reliably tell when someone has had an LLM generate their D&D character, as it’ll always be a half-elf wizard named Erastus.[1]
This is a relatively minor example, I agree that for most cases it doesn’t really matter because the AI randomness is usually not a feature. I do think ‘people have a strong default belief that EU mandates around technology will be bad’ is really quite reasonable of them in general, though, even if in this particular case the effects are minimal.
- ^
I have actually recently had this problem while trying to get LLMs to design and populate a D&D megadungeon for me. I am suspicious that I may have now discovered why.
- ^
This seems very reasonable to me, and I think framing it as ‘abandon a correct answer’ is...not wrong, but a deliberately aggressive framing.
I use AI code assistant tools. It makes a lot of sense to me that an AI code assistant tool whose user disagrees with them should try to figure out whether the user is in fact a good programmer who might know better than them or not. If the AI code assistant will back down when I tell it that it’s wrong about something, but will stand up to a pure vibe-coder with no idea what they’re talking about, that seems like the best possible behavior?
What subset of undergraduates exactly are you engaging with? My own college experience is that the set of undergraduates who would cheerfully get into politics-adjacent arguments with the professors are heavily tilted towards certain views.
2 stones is a substantial handicap at high levels. The saying I recall from the pre-Go-AI days was that a Go grandmaster would take a three-stone handicap to play against God.
Say my dog bites someone, and they need stitches.
It is definitely reasonable to say that I am civilly liable for their medical bills.
It is plausibly reasonable to impose some pain-and-suffering damages, or to accuse me of negligence, especially if my dog has bitten people before.
It seems to me frankly deranged to say that this should be treated legally as if I myself had intentionally bitten the victim.
I appreciate that AI looks quite worrying, but I do not think that “okay, banning AI development directly looks like a hard political sell, but maybe we can find a sneaky way to pervert liability law and make it de facto impossible” is the sort of thought process that leads to good outcomes.
In existing law, this generally applies to civil liability, not criminal. If my farm animals wander onto my neighbor’s property and cause damage, I am civilly responsible for damages. I am NOT criminally responsible as if I had trespassed and intentionally caused the damage myself.
This post is mostly reasonable if I read it as proposing a civil-liability standard. Unfortunately, it really sounds like it’s proposing a criminal-liability standard, which does not seem to me like a reasonable approach.
Very poor use of human capital? If you do a good job of directing skilled people to high-impact jobs, you get better performance in places like software but worse performance in places like convenience stores.
I find it somewhat alarming that the GDPR is given as an example of a good legislation that just needs a bit more enforcement.
Is Thailand setting up to use a gene drive against invasive blackfin tilapia?
aphyer’s Shortform
Seconded. On my end I feel like I see two different kinds of people who frustrate me in AI risk conversations:
People who deny the risk exists at all. Since doom is ~zero probability, there’s no reason to do anything at all about it, and we can get upset at anyone who talks about it for focusing on sci-fi nonsense and distracting from [whatever minor AI effect happens to annoy them].
People who claim that the risk is so great, and doom so certain absent drastic action, that the question of whether the stuff they are doing is stupid and counterproductive is irrelevant. Since doom is ~certain, there’s no way their genius plan could make things worse—the default state is guaranteed extinction, and literally anything is an improvement!
And I feel like P(doom) does a good job of capturing the thing I care about, even if it misses nuances.
Recently a coworker asked me to remote in late after work to help with something.
This inconvenienced me. Yet nevertheless this was a reasonable thing for them to do! The thing was worth me remoting in late for.Then they said ‘sorry to have bothered you with this so late’ or words to that effect. That seems to me like an extraordinarily reasonable thing to say? Are they committing to forevermore conduct themselves in such a way as to never again need to ask me for something after work? No. But frankly that would be kind of a silly thing for me to expect. They are acknowledging that they have inconvenienced me. I appreciate this.
And quotes like these:
...it seems like the smallest ask I could reasonably make in exchange for my forgiveness is that they not do that again...
If I can’t ask not to be hurt again, what can I ask for? Money? Chocolate? Their car?
seem...uh...very weird.
Are you really saying that three examples I cited above are exactly the same?
Not all of them, but at least one. Let’s take the quarantine one first because that’s the one where I think you do well, and then look at the headlines one next. (The AI one doesn’t have enough detail for me to be clear on what’s going on there).
--------------------------------------------------------
I think you’re pretty much on the money for the quarantine one, because:
You are pointing to a relatively-large problem that many people are genuinely unaware of. (Before reading your post I had...uh...some vague idea that some people on a cruise ship had some virus).
You aren’t proposing any working solution—but you’re clear that you don’t have a proposed solution, and also aren’t proposing anything actively harmful.
I do think you may be missing some important points in your quarantine commentary. Like, yes. There are some potential major beneficial effects of these potential carriers being quarantined. By. Uh. Some international organization that can unilaterally detain citizens of multiple different sovereign nations overseas without trial and prevent them from returning home. That seems to me...potentially fraught.
But (as you say), you aren’t proposing solutions at that level of granularity, and you’re being up-front about that. You’re pointing to a situation that looks bad and saying “is there anything we can do about this?” Maybe, on reflection, the answer to this will end up being “it’s going to be a very big diplomatic effort to get even small steps on this problem, and it doesn’t seem likely to succeed.” But that’s not your fault, and everything you have said on this is reasonable.
--------------------------------------------------------On the other hand, the headlines point I think is a much less encouraging one for your view.
Lots of headlines are misleading. What can we do about this? Should we go harass journalists about it? Or perhaps we should go harass editors about it? Those are both proposed solutions at a fair level of granularity. You...seem to me to rather be suggesting both of them. And both of them are (at least in my opinion) bad ideas.
The reason why headlines are misleading is because these businesses need to attract readers in order to continue to exist, and readers demand exaggerated headlines. Talking about which particular employees of a newspaper you should personally harass in order to improve the accuracy of the headlines is like talking about which particular employees of Walmart you should personally harass in order to make them stop selling Twinkies and start selling healthier food.
Here, I think that:
You are pointing to a relatively-small problem that everyone reading your post is already aware of.
You are proposing a solution that seems to me to be actively bad.
and so I think that your overall net score here does not come out positive.
--------------------------------------------------------
Moreover, are you really saying that this generalizes further such that you can never identify a problem without proposing an end-to-end solution?
Not quite. I’m saying that the value of your contribution has two elements:
Pointing to a problem. You gain points here by pointing out real, large problems that your readers are not already familiar with.
Suggesting a solution. You gain points here by suggesting reasonable solutions, and lose points here by suggesting bad solutions.
If Alice points out a substantial new problem, without particularly proposing a detailed solution beyond ‘can we do something about this’, and people are responding by nitpicking issues like that, I agree that’s bad.
But I think it’s a vastly more common situation for Alice to point out a problem that everyone is already aware of and propose an actively bad solution. And I think that the correct response to that looks almost identical to the nitpicking that annoys you.
“This problem is real” is not itself a useful insight. It is useful to the extent that it might lead to the problem being fixed. And fixing problems is much, much, much harder than identifying them.
Perhaps you have noticed that Bay Area rents are really high. This is, indeed, a problem. But noticing that this is a problem is not a serious contribution to the conversation. If you have successfully identified a real problem, and then proposed a solution that will make it worse, you are not helping on net.
This goes double if someone points out that your solution will make it worse, and you say that you “aren’t even primarily interested in fault analysis at [that] level of granularity” and that them pointing that out will “distract you from the problem being real.”
A: “This is a bad problem! We should solve this problem by giving Group X more power to fix it!”
B: “Actually, it sure looks like this problem is plausibly caused by Group X, and certainly they’re exercising all the power they currently have to make it worse. I’m not sure what you hope to accomplish by giving them more power.”
A: “Bad problems don’t stop being bad just because someone is bad at fault analysis!”
B: “No, they don’t, but listening to your solutions to those bad problems can stop being a good idea because you’re bad at fault analysis.”
I think it’s fairly clearly a small joke.
Slowing down AI progress is not itself valuable.
Slowing down AI progress is valuable if and only if you can slow down AI progress without slowing down AI safety progress.
I have faith in the ability of the US government to put lots of red tape and roadblocks in the way of AI development, require that huge armies of lawyers and compliance officers be deployed by any entity trying to develop AI, etc.
I have no faith whatsoever that this process will not also slow down AI safety progress, and some strong suspicion that it might disproportionately slow down AI safety progress.
Overall I think that, when I look at the current conduct of Anthropic, and the current conduct of the US government, I do not find myself saying ‘oh, if only we had given the US government more control over AI companies!’
The last of these seems very unlike the others? We have a list of potential alignment failures, bypassing safeguards, etc...and then one time when the model carelessly deleted the wrong thing. Is the researcher still really mad at Claude about that or something?
I have a complaint. My perfectly-reasonable request to the LLM was denied for reasons of censorship. Communist censorship! This is literally 1984!
The chess community has not made a decision that every person in the country ought to be a chessmaster, and has not attempted to lower its standards until anyone who can fog a mirror is called a GM. If they had, I think you would see similar issues there.