I am Andrew Hyer, currently living in New Jersey and working in New York (in the finance industry).
aphyer
Karma: 5,703
“The methods of justice on which our civilization is built on” contain, as a rather important foundational element, the part where you make something illegal first and then punish people for it only if they keep doing it afterwards.
As someone who thinks that AI risk exists but is skeptical that government involvement will make things worse rather than better, I find the fact that the AI safety movement’s standard-bearer in Washington seems to be Bernie Sanders rather discouraging.
The flip side of this is the tendency to use investigations as a pretext to dig through everything your target has ever done looking for anything (relevant or not) that you might be able to weaponize against them.[1]
“I suspected my old enemy of murdering Dean while he was on their property. After looking through all their security footage, and all their emails, and all their phone call records, and all their business documents, I have discovered incriminating evidence!”
“What, you have evidence that your enemy murdered Dean?”
”Not as such, no. But I have evidence that, on the 8th of June 2024, they described one of their products as being bad but then went on to sell it to customers anyway! And on the 11th of September 2025 they received a report describing how many turtles were killed by their plastic straws, but then on the 8th of October they went on to tell reporters that there was no such effect!”“...does...that...have anything to do with the murder investigation? Why are you even looking at this?”
”I can’t believe you would defend a customer-ripping-off, turtle-murdering monster!”- ^
Wasn’t this actually a plot point in House of Cards?
- ^
The chess community has not made a decision that every person in the country ought to be a chessmaster, and has not attempted to lower its standards until anyone who can fog a mirror is called a GM. If they had, I think you would see similar issues there.
Does this make LLMs much more deterministic? Suppose I ask an LLM to generate a D&D character for me, and do this several hundred times to make a bunch of random NPC adventurers.
An unmodified LLM might make some random decisions and produce a hundred different D&D characters.
A watermarked LLM will watermark itself by reliably producing a half-elf wizard named Erastus, and will produce the same character one hundred times. This will, indeed, let you reliably tell when someone has had an LLM generate their D&D character, as it’ll always be a half-elf wizard named Erastus.[1]
This is a relatively minor example, I agree that for most cases it doesn’t really matter because the AI randomness is usually not a feature. I do think ‘people have a strong default belief that EU mandates around technology will be bad’ is really quite reasonable of them in general, though, even if in this particular case the effects are minimal.
- ^
I have actually recently had this problem while trying to get LLMs to design and populate a D&D megadungeon for me. I am suspicious that I may have now discovered why.
- ^
This seems very reasonable to me, and I think framing it as ‘abandon a correct answer’ is...not wrong, but a deliberately aggressive framing.
I use AI code assistant tools. It makes a lot of sense to me that an AI code assistant tool whose user disagrees with them should try to figure out whether the user is in fact a good programmer who might know better than them or not. If the AI code assistant will back down when I tell it that it’s wrong about something, but will stand up to a pure vibe-coder with no idea what they’re talking about, that seems like the best possible behavior?
What subset of undergraduates exactly are you engaging with? My own college experience is that the set of undergraduates who would cheerfully get into politics-adjacent arguments with the professors are heavily tilted towards certain views.
2 stones is a substantial handicap at high levels. The saying I recall from the pre-Go-AI days was that a Go grandmaster would take a three-stone handicap to play against God.
Say my dog bites someone, and they need stitches.
It is definitely reasonable to say that I am civilly liable for their medical bills.
It is plausibly reasonable to impose some pain-and-suffering damages, or to accuse me of negligence, especially if my dog has bitten people before.
It seems to me frankly deranged to say that this should be treated legally as if I myself had intentionally bitten the victim.
I appreciate that AI looks quite worrying, but I do not think that “okay, banning AI development directly looks like a hard political sell, but maybe we can find a sneaky way to pervert liability law and make it de facto impossible” is the sort of thought process that leads to good outcomes.
In existing law, this generally applies to civil liability, not criminal. If my farm animals wander onto my neighbor’s property and cause damage, I am civilly responsible for damages. I am NOT criminally responsible as if I had trespassed and intentionally caused the damage myself.
This post is mostly reasonable if I read it as proposing a civil-liability standard. Unfortunately, it really sounds like it’s proposing a criminal-liability standard, which does not seem to me like a reasonable approach.
Very poor use of human capital? If you do a good job of directing skilled people to high-impact jobs, you get better performance in places like software but worse performance in places like convenience stores.
I find it somewhat alarming that the GDPR is given as an example of a good legislation that just needs a bit more enforcement.
Is Thailand setting up to use a gene drive against invasive blackfin tilapia?
aphyer’s Shortform
Seconded. On my end I feel like I see two different kinds of people who frustrate me in AI risk conversations:
People who deny the risk exists at all. Since doom is ~zero probability, there’s no reason to do anything at all about it, and we can get upset at anyone who talks about it for focusing on sci-fi nonsense and distracting from [whatever minor AI effect happens to annoy them].
People who claim that the risk is so great, and doom so certain absent drastic action, that the question of whether the stuff they are doing is stupid and counterproductive is irrelevant. Since doom is ~certain, there’s no way their genius plan could make things worse—the default state is guaranteed extinction, and literally anything is an improvement!
And I feel like P(doom) does a good job of capturing the thing I care about, even if it misses nuances.
Recently a coworker asked me to remote in late after work to help with something.
This inconvenienced me. Yet nevertheless this was a reasonable thing for them to do! The thing was worth me remoting in late for.Then they said ‘sorry to have bothered you with this so late’ or words to that effect. That seems to me like an extraordinarily reasonable thing to say? Are they committing to forevermore conduct themselves in such a way as to never again need to ask me for something after work? No. But frankly that would be kind of a silly thing for me to expect. They are acknowledging that they have inconvenienced me. I appreciate this.
And quotes like these:
...it seems like the smallest ask I could reasonably make in exchange for my forgiveness is that they not do that again...
If I can’t ask not to be hurt again, what can I ask for? Money? Chocolate? Their car?
seem...uh...very weird.
Are you really saying that three examples I cited above are exactly the same?
Not all of them, but at least one. Let’s take the quarantine one first because that’s the one where I think you do well, and then look at the headlines one next. (The AI one doesn’t have enough detail for me to be clear on what’s going on there).
--------------------------------------------------------
I think you’re pretty much on the money for the quarantine one, because:
You are pointing to a relatively-large problem that many people are genuinely unaware of. (Before reading your post I had...uh...some vague idea that some people on a cruise ship had some virus).
You aren’t proposing any working solution—but you’re clear that you don’t have a proposed solution, and also aren’t proposing anything actively harmful.
I do think you may be missing some important points in your quarantine commentary. Like, yes. There are some potential major beneficial effects of these potential carriers being quarantined. By. Uh. Some international organization that can unilaterally detain citizens of multiple different sovereign nations overseas without trial and prevent them from returning home. That seems to me...potentially fraught.
But (as you say), you aren’t proposing solutions at that level of granularity, and you’re being up-front about that. You’re pointing to a situation that looks bad and saying “is there anything we can do about this?” Maybe, on reflection, the answer to this will end up being “it’s going to be a very big diplomatic effort to get even small steps on this problem, and it doesn’t seem likely to succeed.” But that’s not your fault, and everything you have said on this is reasonable.
--------------------------------------------------------On the other hand, the headlines point I think is a much less encouraging one for your view.
Lots of headlines are misleading. What can we do about this? Should we go harass journalists about it? Or perhaps we should go harass editors about it? Those are both proposed solutions at a fair level of granularity. You...seem to me to rather be suggesting both of them. And both of them are (at least in my opinion) bad ideas.
The reason why headlines are misleading is because these businesses need to attract readers in order to continue to exist, and readers demand exaggerated headlines. Talking about which particular employees of a newspaper you should personally harass in order to improve the accuracy of the headlines is like talking about which particular employees of Walmart you should personally harass in order to make them stop selling Twinkies and start selling healthier food.
Here, I think that:
You are pointing to a relatively-small problem that everyone reading your post is already aware of.
You are proposing a solution that seems to me to be actively bad.
and so I think that your overall net score here does not come out positive.
--------------------------------------------------------
Moreover, are you really saying that this generalizes further such that you can never identify a problem without proposing an end-to-end solution?
Not quite. I’m saying that the value of your contribution has two elements:
Pointing to a problem. You gain points here by pointing out real, large problems that your readers are not already familiar with.
Suggesting a solution. You gain points here by suggesting reasonable solutions, and lose points here by suggesting bad solutions.
If Alice points out a substantial new problem, without particularly proposing a detailed solution beyond ‘can we do something about this’, and people are responding by nitpicking issues like that, I agree that’s bad.
But I think it’s a vastly more common situation for Alice to point out a problem that everyone is already aware of and propose an actively bad solution. And I think that the correct response to that looks almost identical to the nitpicking that annoys you.
“This problem is real” is not itself a useful insight. It is useful to the extent that it might lead to the problem being fixed. And fixing problems is much, much, much harder than identifying them.
Perhaps you have noticed that Bay Area rents are really high. This is, indeed, a problem. But noticing that this is a problem is not a serious contribution to the conversation. If you have successfully identified a real problem, and then proposed a solution that will make it worse, you are not helping on net.
This goes double if someone points out that your solution will make it worse, and you say that you “aren’t even primarily interested in fault analysis at [that] level of granularity” and that them pointing that out will “distract you from the problem being real.”
A: “This is a bad problem! We should solve this problem by giving Group X more power to fix it!”
B: “Actually, it sure looks like this problem is plausibly caused by Group X, and certainly they’re exercising all the power they currently have to make it worse. I’m not sure what you hope to accomplish by giving them more power.”
A: “Bad problems don’t stop being bad just because someone is bad at fault analysis!”
B: “No, they don’t, but listening to your solutions to those bad problems can stop being a good idea because you’re bad at fault analysis.”
“New legal ground may be required”
“Standard defenses are pre-emptively rejected”
“Existing institutions are inadequate or complicit”
...yyyeah, I’m sorry, if this seems reasonable-ish to you I do not think I want you to come anywhere near political power, and I don’t even work at an AI lab.