Jono

Karma: 29

AI demands unprecedented reliability

Jono9 Jan 2024 16:30 UTC

22 points

5 comments2 min readLW link

Jono 29 Nov 2023 9:51 UTC
3 points
1
in reply to: Alex_Altair’s comment on: Shallow review of live agendas in alignment & safety
ai-plans.com aims to collect research agendas and have people comment on their strengths and vulnerabilities. The discord also occasionally hosts a critique-a-ton, where people discuss specific agendas.

Jono 27 Oct 2023 9:47 UTC
LW: 1 AF: 5
−9
AF
in reply to: TurnTrout’s comment on: AI as a science, and three obstacles to alignment strategies
We do not know, that is the relevant problem.
Looking at the output of a black box is insufficient. You can only know by putting the black box in power, or by deeply understanding it.
Humans are born into a world with others in power, so we know that most humans care about each other without knowing why.
AI has no history of demonstrating friendliness in the only circumstances where that can be provably found. We can only know in advance by way of thorough understanding.
A strong theory about AI internals should come first. Refuting Yudkowsky’s theory about how it might go wrong is irrelevant.

Jono 8 Sep 2023 8:28 UTC
0 points
0
on: The Löbian Obstacle, And Why You Should Care
Layman here 👋
Iiuc we cannot trust the proof of an unaligned simulacra’s suggestion because if it is smarter than us.
Would that be a non-issue if verifying the proof is easier than making it?
If we can know how hard it is to verify a proof without verifying, then we can find a safe protocol for communicating with this simulacra. Is this possible?

Jono 29 Sep 2022 15:10 UTC
2 points
0
on: existential self-determination
it might be the case that any kind of meaningful values would be reasonably encodable as answers to the question “what next set of MPIs should be instantiated?”
What examples of (meaningless) values are not answers to “What next set of MPIs should be instantiated?”

Jono 8 Sep 2022 16:19 UTC
2 points
0
on: All AGI safety questions welcome (especially basic ones) [~monthly thread]
What does our world (a decade) after the employment of a successfully aligned AGI look like?

Jono 10 Jun 2022 6:50 UTC
1 point
0
in reply to: plex’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Thank you plex, I was not aware of this wiki.
The pitch is nice, I’ll incorporate it.

Jono 10 Jun 2022 6:40 UTC
7 points
5
in reply to: Carl Feynman’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
Why do I care if the people around me care about AI risk?

1. when AI is going to rule we’d like the people to somehow have some power I reckon.
I mean creating any superintelligence is a powergrab. Making one in secret is quite hostile, shouldn’t people get a say or at least insight in what their future holds?
2. Nobody still really knows what we’d like the superint to do. I think an ML researcher is as capable of voicing their desires for the future as an artist. The field surely can benefit from interdisciplinary approaches.
3. As with nuclear war, I’m sure politicians will care more when the people care more. AI governance is a big point. Convincing AI devs to not make the superint seems easier when a big percentage of humanity is pressuring them not to do it.
4. Maybe this also extends to international relations. Seeing that the people of a democratic country care about the safety, makes the ventures from that country seem more reliable.
5. I get bummed out when nobody knows what I’m talking about.

Jono 9 Jun 2022 13:31 UTC
1 point
0
in reply to: Yonatan Cale’s comment on: AGI Safety FAQ / all-dumb-questions-allowed thread
I’m concerned with the ethics.
Is it wrong to doom speak to strangers? Is that the most effective thing here? I’d be lying if I said I was fine, but would it be best to tell them I’m “mildly concerned”?
How do convey these grave emotions I have while maximally getting the people around me to care about mitigating AI risk?
Should I compromise on truth and downplay my concerns if that will get someone to care more? Should I expect people to be more receptive to the message of AI risk if I’m mild about it?

Jono 9 Jun 2022 7:05 UTC
2 points
2
on: AGI Safety FAQ / all-dumb-questions-allowed thread
What do I tell the people who I know but can’t spend lots of time with?
Clarification: How do I get relative strangers who converse with me IRL to maximally care about the dangers of AI?
Do I downplay my concerns such that they don’t think I’m crazy?
Do I mention it every time I see them to make sure they don’t forget?
Do I tolerate third parties butting in and making wrong statements?
Do I tell them to read up on it and pester them on whether they read it already?
Do I never mention it to laymen to avoid them propagating wrong memes?
Do I seek out and approach people who could be useful to the field or should I just forward or mention them to someone who can give the topic a better first impression than me?