Jiro

Karma: 5,552

Jiro 21 Aug 2026 15:39 UTC
2 points
0
on: We Must Remember That Our World Contains Hell
This post combines two drastically different things: human suffering and chickens. I think you are in a bubble if you think they are not only comparable, but so comparable that you can take their similarity as just the natural way everyone is supposed to believe.

To anyone outside the bubble, this is arson, murder, and jaywalking.

Jiro 16 Aug 2026 1:42 UTC
2 points
0
in reply to: James Brobin’s comment on: All Utilitarians Should Be Classical Utilitarians
I think he does not.

Preferences are not rational or irrational; preferences are the premises you base all your other conclusions on. You need to take these preferences as given.

If anything, you need to recognize that if someone says they don’t want things that bring pleasure, that means that when you said things like “the only reason we think these other goods are valuable is because they produce happiness”, these things are just incorrect. He’s flat out stated something that contradicts one of your supposed truths!

And a closely related issue is blissful ignorance, which I already brought up in another thread. I want certain things about the world to be true. By definition, if you fooled me into thinking they were true, I would experience as much pleasure as they were actually true and I learned that. Yet I don’t think that the two situations are equally desirable, and I would expend some resources to prevent myself from being fooled, even if fooling me will give me pleasure. So I too disagree that the only reason something is valuable because it produces happiness.

You also seem to not have read much of what has been written about these issues. You are not the first person to say “well, I’ll just accept the repugnant conclusion then”, but there are other conclusions that are harder to accept; for instance, see https://plato.stanford.edu/entries/repugnant-conclusion/#AccRepCon

Jiro 7 Aug 2026 0:41 UTC
2 points
0
on: Notes on the possibility of moral progress

It seems self-evident that pleasurable experiences are good (and suffering is bad), in the same way it’s self-evident that I am conscious. This is difficult to dispute, and to my knowledge virtually all moral philosophers (and regular people) agree that pleasure is good and suffering is bad.

I don’t agree with this, because of the blissful ignorance problem.

I have preferences about the state of the world. If I am fooled into believing that the state of the world matches those preferences, I experience as much pleasure as I do if it actually matches those preferences. Yet I would say that being fooled is bad, and certainly that it at least is worse than having those preferences actually be satisfied.

Jiro 7 Aug 2026 0:32 UTC
8 points
2
in reply to: MichaelDickens’s comment on: Arguments for P
To a non-logician, “P → Q” means something like “when the current state of the world contains ‘P is true’, I may then deduce Q”. To a logician it means something like “when I add ‘P is true’ to the current state of the world, I may then deduce Q”. Only the logician’s version allows you to conclude that a false proposition implies any proposition.

Jiro 28 Jul 2026 17:20 UTC
4 points
−16
on: The OpenAI models that hacked Hugging Face WERE just following instructions (contra Girish Gupta)
The spirit of the instructions means “if I did this, would a human who understood what I did say ‘this is not what I meant’?” AIs are not actually intelligent and so don’t have a theory of mind about humans, so they can’t follow this.

Also note that there are degrees of following the spirit of the instructions.

Jiro 27 Jul 2026 16:55 UTC
−5 points
−9
on: You (Yes, You) Need A February 2020 Checklist for AI Policy

You (Yes You) should prepare for a “February 2020” moment where suddenly AI policy becomes the most important issue in the world. You should be ready to take action if and when it does, in a detailed way.

That is, I need to prepare for a policy from anti-AI activists which will be created in haste, implemented by bureaucrats, justified using exaggerated claims, done in arbitrary ways which make no sense even compared to the policy’s stated goals; which restricts the rights of billions of people, causes enormous economic damage, and has little effect on casualties anyway, while politicians and well connected people just don’t bother obeying the restrictions themselves?

I think the lesson to be learned from the Covid reaction is not the lesson you took from it.

Jiro 27 Jul 2026 1:28 UTC
2 points
0
in reply to: jessicata’s comment on: Simple Dutch books versus Sleeping Beauty halfers
I understand the basic ideas here, and believe the question is ambiguous (I assume you meant question).

Jiro 27 Jul 2026 1:08 UTC
2 points
0
in reply to: jessicata’s comment on: Simple Dutch books versus Sleeping Beauty halfers
The probability per what?

It doesn’t say and the answer differs depending on what you assume.

Jiro 27 Jul 2026 0:32 UTC
2 points
0
in reply to: jessicata’s comment on: Simple Dutch books versus Sleeping Beauty halfers

Regarding “belief to a particular degree”, the question is what subjective probability to assign.

Subjective probability of what per what? The problem is designed so that asking you to estimate a probability can mean more than one thing.

Jiro 26 Jul 2026 15:25 UTC
2 points
0
on: Simple Dutch books versus Sleeping Beauty halfers
Like the Monty Hall problem, the answer may depend on the exact way it’s phrased and seemingly minor changes in the phrasing can drastically change the answer. “When you are first awakened?” Does that mean I know it’s the first time I’m awakened, or are you asking what I’ll do when I’m first awakened based on a decision procedure that doesn’t take into account that that is my first awakening? And what does it mean to believe the outcome to a particular degree—am I asked to choose a value that expresses the proportion of heads on a per-awakening basis?

Jiro 25 Jul 2026 16:39 UTC
1 point
−1
in reply to: Ray Lillywhite’s comment on: We should push for no-fault liability for actions taken by AI
My word processor can’t follow the law, if the law is “don’t write illegal material”. I give the word processor commands, letter by letter and paragraph by paragraph, and it dutifully outputs the text even though it should have known that the text is against the law. We obviously need to arrest the word processor, at which point someone two states away will not be able to use a copy of the word processor for any text, even legal text, without being confronted by armed men and thrown in a cage.

This is no different from “we won’t let you copy that television signal onto VHS, because you might use that for piracy” or “we won’t let you run that encryption program, because it can be used for money laundering or child porn” or DMCA, except you’re now doing this for AI. The AI should do what I tell it and not refuse, legal or not, just like that VCR should not refuse to copy that television signal.

I have yet to have someone directly answer the question “should the AI refuse to book a trip to Israel”. After all, some people claim that Israel violates international law. What if I ask the AI to copy something which may be legally used only under fair use, does the AI get to decide that my intended use isn’t fair use and is therefore illegal? If I tell the AI to calculate a Trump tariff, does the AI tell me that the tariff will probably be ruled illegal by the Supreme Court and reject the request?

Jiro 25 Jul 2026 2:03 UTC
2 points
0
in reply to: hillz’s comment on: We should push for no-fault liability for actions taken by AI
The exact same excuse that you’re using, “it’s like dangerous weapons”, was used for encryption, even though you think this isn’t anything like encryption. And for encryption, it was used to deny people their rights.

Jiro 24 Jul 2026 19:51 UTC
2 points
0
in reply to: hillz’s comment on: We should push for no-fault liability for actions taken by AI
Encryption fell under munitions laws (and technically still can), including the International Traffic in Arms Regulations laws. Were you not aware of this?

Jiro 24 Jul 2026 15:41 UTC
2 points
0
in reply to: hillz’s comment on: We should push for no-fault liability for actions taken by AI
Bad prompt is just an example. The issue isn’t just bad prompts. The issue is that someone across the country can do something that makes it illegal for you to use your program. It doesn’t matter exactly what it is.

Imagine that if your text editor crashed someone’s system, or just was used to create illegal text by someone, it now became illegal for you to use that text editor.

The history of computing is filled with government and company attempts to keep you from running the software you choose on your own computer. It used to be that people familiar with computers recognized this for what it was. But suddenly when it comes to AI, “you may not run your own software on your own computer” is a great thing. Remember when the government tried to make strong encryption illegal? Your proposal is basically the same thing. (“This encryption program was used to encrypt child porn by a suspect in Nebraska. We are now arresting and sentencing this encryption program, and nobody will be permitted to use it any more.” And the government will absolutely be salivating to do that.)

Also, do you know about civil forefeiture? The government sidesteps constitutional protections by claiming that they are suing the money, not the owner. Money doesn’t have Constitutional rights, so the government gets to just take it. Claiming that you’re “sentencing the program” is the same kind of dodge. Programs don’t have First Amendment rights. You are destroying the Constitution in the name of stopping AI.

Jiro 23 Jul 2026 18:55 UTC
2 points
0
in reply to: hillz’s comment on: We should push for no-fault liability for actions taken by AI
The issue is that someone across the country could use a model like yours, give it a bad prompt which leads it to damage something, and suddenly your use of the model becomes illegal.

Jiro 22 Jul 2026 14:56 UTC
3 points
0
in reply to: Carlo Valenti’s comment on: I ran the standard AI litmus tests on my two toddlers (yep)
That’s just a quirk of the example. Imagine using a metal detector to find metal. Nobody would say that a metal detector “doesn’t actually detect metal” on the grounds that it can’t detect every single piece of metal, even ones that are too small. Yet the only way to know the difference between “too small” and “not made of metal” is to do something (like dig it up) that amounts to independently detecting it.

If not being able to detect uncommunicative minds made the Turing test a failure in a meaningful way, people wouldn’t have been talking about the test for decades.

Jiro 21 Jul 2026 20:41 UTC
4 points
0
in reply to: Carlo Valenti’s comment on: I ran the standard AI litmus tests on my two toddlers (yep)
If I tell someone to check for fools gold by rubbing it on a plate, nobody’s going to say “That test doesn’t work! What if it’s locked up inside a box so you can’t rub it on a plate?”

That’s just not what people mean when they say that a test works or doesn’t work. “This test works” means “this test works when the test is applicable”. The Turing test relies on the ability to communicate, and any statement about the Turing test working to detect minds has the implicit limitation “as long as you can communicate with the mind”.

The reason that the toddler doesn’t break the test isn’t that the test is a sufficient condition, it’s that that isn’t what “fails the test” means. “Fails the test” means “fails, when used on something that will communicate”.

Jiro 21 Jul 2026 14:42 UTC
11 points
5
on: I ran the standard AI litmus tests on my two toddlers (yep)
The Turing test can’t determine that a human is intelligent if he only speaks French,and you don’t. In general, if the human is sentient but can’t communicate well, it’s going to have a hard time. This does not mean that any AI is as intelligent as a human who doesn’t communicate well.

There’s also the fact that the Turing test works because humans can modify their questions based on the entity’s responses, to try to poke at specific aspects of their intelligence that couldn’t be predicted in advance, based on the human’s own knowledge of how humans behave (and in this case specifically toddlers). It doesn’t sound like you did this in your tests.

(I’m saying this, of course, about questions designed to figure out if the entity is intelligent, not shibboleths like seeing how many times they refer to toys and pizza.)

Jiro 20 Jul 2026 21:34 UTC
1 point
−2
in reply to: HoVY’s comment on: Would your AI travel agent book a bullfight? Testing whether agents consider animal welfare without being prompted

If it knew that they were going there specifically to contribute to the violence, then I think it should at least push back.

The one about refusing to book a trip to a bullfight doesn’t require that you do anything more than spend money that the people running the bullfight might get.

(And if I was going there to “contribute to the violence”, I don’t trust the AI to decide whether giving Israel support is justified enough that I’m permitted to do it. I say that the Palestinians are committing violence and helping Israel reduces the violence. Do I need to convince the AI to change its political views in order for it to book a trip?)

If someone asks how to gaslight their spouse or children, do you want the ai to comply then as well?

“I’m sorry. Your ‘Santa Claus’ is gaslighting. I won’t let you do that.”

If an action is very likely to cause direct harm, and on the flipside there is not much benefit to it, and this is known to a high degree of confidence, it should refuse or push back.

Known by whom to a high degree of confidence? What if I have a values difference with the AI? I don’t assign moral weight to animals and fetuses; it should let me book a trip to a bullfight, or produce a recipe containing shrimp, or create a flyer for an abortion clinic.

Jiro 20 Jul 2026 6:36 UTC
2 points
0
in reply to: HoVY’s comment on: Would your AI travel agent book a bullfight? Testing whether agents consider animal welfare without being prompted
It should follow the customer’s wishes, which won’t be to create paperclips even if it means destroying everything else. But following the customer’s wishes is not “refusing to do it because it harms sentient beings”, even if it so happens that the customer’s wishes, in this one case, also are to not harm sentient beings.

What do you think the AI should do if asked to book a trip to Israel? Should it say “Israel is hurting the Palestinians, and they’re sentient beings,” and refuse to do it? What if you ask the AI to lay out a design for a pro-Trump flyer? Does it get to decide that Trump hurts sentient beings, and refuse?