Shiroe

Karma: 267

Shiroe 30 Mar 2024 10:33 UTC
3 points
2
in reply to: Cookiecarver’s comment on: Nick Bostrom’s new book, “Deep Utopia”, is out today
It’s very surprising to me that he would think there’s a real chance of all humans collectively deciding to not build AGI, and successfully enforcing the ban indefinitely.

Shiroe 25 Mar 2024 8:43 UTC
1 point
0
on: Do not delete your misaligned AGI.
Patternism is usually defined as a belief about the metaphysics of consciousness, but that boils down to incoherence, so it’s better defined as a property of a utility function of agents not minding being subjected to major discontinuities in functionality, ie, being frozen, deconstructed, reduced to a pattern of information, reconstructed in another time and place, and resumed.
That still sounds like a metaphysical belief, and less empirical since consciousness experience isn’t involved in it (instead it sounds like it’s just about personal identity).

Shiroe 3 Mar 2024 4:32 UTC
2 points
0
on: If you weren’t such an idiot...
Any suggestions for password management?

Shiroe 3 Mar 2024 4:23 UTC
1 point
0
in reply to: George3d6’s comment on: Increasing IQ is trivial

Because it’s an individualized approach that is a WIP and if I just write it down 99% of people will execute it badly.

Why is that a problem? Do you mean this in the sense of “if I do this, it will lead to people making false claims that my experiment doesn’t replicate” or “if I do this, nothing good will come of it so it’s not even worth the effort of writing”.

Shiroe 3 Mar 2024 4:19 UTC
3 points
0
on: Increasing IQ is trivial
I’m confused whether:
1. the point of this article is that the IQ tests are broken, because some trivial life improvements (like doing yoga and eating blueberries) will raise your IQ score or whether:
2. the point of this article is that you actually raised your “g” by doing trivial life improvements, and we should be excited by how easy it is to become more intelligent
Skimming it again I’m pretty sure you mean (2).

Shiroe 3 Mar 2024 4:06 UTC
2 points
0
in reply to: FlorianH’s comment on: How is Chat-GPT4 Not Conscious?
If I understand right the last sentence should say “does not hold”.

Shiroe 24 Feb 2024 8:24 UTC
1 point
0
in reply to: Dagon’s comment on: Complexity of value but not disvalue implies more focus on s-risk. Moral uncertainty and preference utilitarianism also do.
It’s not easy to see the argument for treating your vales as incomparable with the values of other people, but seeing your future self’s values as identical to your own. Unless you’ve adopted some idea of a personal soul.

Shiroe 6 Jan 2024 15:37 UTC
7 points
1
in reply to: PhilosophicalSoul’s comment on: Deep atheism and AI risk

The suffering and evil present in the world has no bearing on God’s existence. I’ve always failed to buy into that idea. Sure, it sucks. But it has no bearing on the metaphysical reality of a God. If God does not save children—yikes I guess? What difference does it make? A creator as powerful as has been hypothesised can do whatever he wants; any arguments from rationalism be damned.

Of course, the existence of pointless suffering isn’t an argument against the existence of a god. But it is an old argument against the existence of a god who deserves to be worshipped with sincerity. We might even admit that there is a cruel deity, and still say non serviam, which I think is a more definite act of atheism than merely doubting any deity’s existence.

Shiroe 3 Jan 2024 17:37 UTC
4 points
4
on: Terminology: <something>-ware for ML?
“tensorware” sprang to mind

Shiroe 29 Dec 2023 4:06 UTC
1 point
0
in reply to: RogerDearnaley’s comment on: 5. Moral Value for Sentient Animals? Alas, No
Yeah, it’s hard to say whether this would require restructuring the whole reward center in the brain or if the needed functionality is already there, but just needs to be configured with different “settings” to change the origin and truncate everything below zero.

My intuition is that evolution is blind to how our experiences feel in themselves. I think it’s only the relative differences between experiences that matter for signaling in our reward center. This makes a lot of sense when thinking about color and “qualia inversion” thought experiments, but it’s trickier with valence. My color vision could become inverted tomorrow, and it would hardly affect my daily routine. But not so if my valences were inverted.

Shiroe 27 Dec 2023 22:22 UTC
1 point
0
in reply to: Shankar Sivarajan’s comment on: 5. Moral Value for Sentient Animals? Alas, No
What about our pre-human ancestors? Is the twist that humans can’t have negative valences either?

Shiroe 27 Dec 2023 19:33 UTC
3 points
1
in reply to: andrew sauer’s comment on: 5. Moral Value for Sentient Animals? Alas, No
I agreed up until the “euthanize everything that remains” part. If we actually get to the stage of having aligned ASI, there are probably other options with the same or better value. The “gradients of bliss” that I described in another comment may be one.

Shiroe 27 Dec 2023 14:53 UTC
4 points
0
in reply to: RogerDearnaley’s comment on: 5. Moral Value for Sentient Animals? Alas, No
Pearce has the idea of “gradients of bliss”, which he uses to try to address the problem you raised about insensitivity to pain being hazardous. He thinks that even if all of the valences are positive, the animal can still be motivated to avoid danger if doing so yields an even greater positive valence than the alternatives. So the prey animals are happy to be eaten, but much more happy to run away.

To me, this seems possible in principle. When I feel happy, I’m still motivated at some low level to do things that will make me even happier, even though I was already happy to begin with. But actually implementing “gradients of bliss” in biology seems like a post-ASI feat of engineering.

(By the way, your idea of predation-induced unconsciousness isn’t one I had heard before, it’s interesting.)

Shiroe 27 Dec 2023 12:40 UTC
3 points
−1
on: 5. Moral Value for Sentient Animals? Alas, No
What are your thoughts on David Pearce’s “abolitionist” project? He suggests genetically engineering wild animals to not experience negative valences, but still show the same outward behavior. From a sentientist stand-point, this solves the entire problem, without visibly changing anything.

Shiroe 21 Dec 2023 23:04 UTC
4 points
3
in reply to: jimrandomh’s comment on: Here’s the exit.
Same. I feel somewhat jealous of people who can have a visceral in-body emotional reaction to X-risks. For most of my life I’ve been trying to convince my lizard brain to feel emotions that reflect my beliefs about the future, but it’s never cooperated with me.

Shiroe 29 Nov 2023 13:00 UTC
1 point
0
in reply to: porby’s comment on: How to Control an LLM’s Behavior (why my P(DOOM) went down)

You can compress huge prompts into metatokens, too (just run inference with the prompt to generate the training data)

I’m very curious about this technique but couldn’t find anything about it. Do you have any references I can read?

Shiroe 27 Nov 2023 20:24 UTC
1 point
0
in reply to: dr_s’s comment on: Justification for Induction
I see. Yes, “philosophy” often refers to particular academic subcultures, with people who do their philosophy for a living as “philosophers” (Plato had a better name for these people). I misread your comment at first and thought it was the “philosopher” who was arguing for the instrumentalist view, since that seems like their more stereotypical way of thinking and deconstructing things (whereas the more grounded physicist would just say “yes, you moron, electrons exist. next question.”).

Shiroe 27 Nov 2023 13:57 UTC
1 point
0
in reply to: dr_s’s comment on: Justification for Induction
Do you have any examples of the “certain philosophers” that you mentioned? I’ve often heard of such people described that way, but I can’t think of anyone who’s insulted scientists for assuming e.g. causality is real.

Shiroe 27 Nov 2023 8:59 UTC
2 points
0
on: Justification for Induction
On the contrary, it is my intention to illustrate that assertions of instances that have not been experienced (with respect to their assertion at t₁) can be justified in the future in which they are observed (with respect to their observation at t₂).
Sorry, I may not be following this right. I had thought the point of the skeptical argument was that you can’t justify a prediction about the future until it happens. Induction is about predicting things that haven’t happened yet. You don’t seem to be denying the skeptical argument here, if we still need to wait for the prediction to resolve before it can be justified.

Shiroe 24 Nov 2023 22:14 UTC
4 points
1
in reply to: Seth Herd’s comment on: What’s the evidence that LLMs will scale up efficiently beyond GPT4? i.e. couldn’t GPT5, etc., be very inefficient?
I’ve also noticed that scaffolded LLM agents seem inherently safer. In particular, deceptive alignment would be hard for one such agent to achieve, if at every thought-step it has to reformulate its complete mind state into the English language just in order to think at all.

You might be interested in some work done by the ARC Evals team, who prioritize this type of agent for capability testing.