Anon User

Karma: 531

Anon User Jun 2, 2023, 12:54 AM
−1 points
0
on: Value Physics

When an AGI takes on values for the first time, it must draw from the set of values which already exist or construct something similar from what already exists

The values come into the picture well before it’s an AGI. First, a random neural network is initialized, and its “values” is a completely arbitrary function chosen as random. Over time, NN is trained towards an AGI and it’s “values” take shape. By the time AGI emerges, it does not “take on values for the first time”, the values emerge from an extremely long sequence of tiny mutations, each creating something very similar to what already existed, becoming more complex and coherent over time.

Anon User May 27, 2023, 7:17 PM
3 points
1
on: No—AI is just as energy-efficient as your brain.
I made a similar point (but without specific numbers—great to have them!) in a comment https://www.lesswrong.com/posts/Lwy7XKsDEEkjskZ77/?commentId=nQYirfRzhpgdfF775 on a post that posited human brain energy efficiency over AIs as a core anti-doom argument, and I also think that the energy efficiency comparisons are not particularly relevant either way:

Humanity is generating and consuming enormous amount of power—why is the power budget even relevant? And even if it was, energy for running brains ultimately comes from Sun—if you include the agriculture energy chain, and “grade” the energy efficiency of brains by the amount of solar energy it ultimately takes to power a brain, AI definitely has a potential to be more efficient. And even if a single human brain is fairly efficient, the human civilization is clearly not. With AI, you can quickly scale up the amount of compute you use, but scaling beyond a single brain is very inefficient.

Anon User May 27, 2023, 7:05 PM
1 point
0
in reply to: Adrien Sicart’s comment on: Orthogonality Thesis: Proof that Strong Form does not hold.
Well, yeah, if you specifically choose a crippled version of the high-U agent that is somehow unable to pursue the winning strategy, it will loose—but IMHO that’s not what the discussion here should be about.

Anon User May 27, 2023, 7:00 PM
1 point
0
in reply to: ArisC’s comment on: A rejection of the Orthogonality Thesis
And Gordon Seidoh Worley is not saying there can’t be good arguments against orthogonality thesis that would deserve uovotes, just that this one is not one of those.

Anon User May 27, 2023, 6:54 PM
1 point
0
on: A rejection of the Orthogonality Thesis

This line of reasoning is absurd: it assumes an agent knows in advance the precise effects of self-improvement — but that’s not how learning works! If you knew exactly how an alteration in your understanding of the world would impact you, you wouldn’t need the alteration: to be able to make that judgement, you’d have to be able to reason as though you had already undergone it.

It seems there is some major confusion is going on here—it is, generally speaking, imporrible to know the outcome of an arbitrary computation without actually running it, but that does not mean it’s impossible to design a specific computation in a way you’d know exactly what the effects would be. For example, one does not need to know the trillionth digit of pi in order to write a program that they could be very certain would compute that digit.

You also seem to be too focused on minor modifications of a human-like mind, but focusing too narrowly on minds is also missing the point—focus on optimization programs instead.

For many different kinds of X, it should be possible to write a program that given a particular robotics apparatus (just the electromechanical parts without a specific control algorithm), predicts which electrical signals sent to robot’s actuators would result in more X. You can then place that program inside the robot and have the program’s output wired to the robot controls. The resulting robot does not “like” X, it’s just robotically optimizing for X.

The orthogonality principle just says that there is nothing particularly special about human-aligned Xs that would make the X-robot more likely to work well for those Xs over Xs that result in human extinction (e.g. due to convergent instrumental goals, X does not need to specifically be anti-human).

Anon User May 17, 2023, 4:30 PM
1 point
0
on: Orthogonality Thesis: Proof that Strong Form does not hold.
Wait, if Clip-maniac finds itself in a scenario where Clippy would achieve higher U then itself, the rational thing for it would be to self-modify into Clippy, and the Strong Form would still hold, wouldn’t it?

Anon User Apr 24, 2023, 8:53 AM
7 points
3
in reply to: the gears to ascension’s comment on: Contra Yudkowsky on AI Doom
Exactly! I’d expect compute to scale way better than humans—not necessarily because the intelligence of compute scales so well, but because the intelligence of human groups scales so poorly...

Anon User Apr 24, 2023, 12:56 AM
1 point
0
in reply to: Adam Golding’s comment on: Votes-per-Dollar
The advertising has to be visible, but who exactly paid for it does not have to be. And there are plenty of less obvious spending (e.g. paying people to go door-to-door, phone calls, etc, etc—pay people, then claim they were volunteers?).

Anon User Apr 24, 2023, 12:54 AM
16 points
7
on: Contra Yudkowsky on AI Doom
Humanity is generating and consuming enormous amount of power—why is the power budget even relevant? And even if it was, energy for running brains ultimately comes from Sun—if you include the agriculture energy chain, and “grade” the energy efficiency of brains by the amount of solar energy it ultimately takes to power a brain, AI definitely has a potential to be more efficient. And even if a single human brain is fairly efficient, the human civilization is clearly not. With AI, you can quickly scale up the amount of compute you use, but scaling beyond a single brain is very inefficient.
What links here?
- Anon User's comment on No—AI is just as energy-efficient as your brain. by Maxwell Clarke (May 27, 2023, 7:17 PM; 3 points)

Anon User Apr 17, 2023, 7:20 PM
3 points
2
on: Prediction: any uncontrollable AI will turn earth into a giant computer
Temporal discounting is a thing—not sure why you are certain an ASI would not have enough temporal discounting in its value function to be unwilling to delay gratification by so much.

Anon User Apr 15, 2023, 8:50 PM
5 points
1
on: [linkpost] “What Are Reasonable AI Fears?” by Robin Hanson, 2023-04-23

Doomers worry about AIs developing “misaligned” values. But in this scenario, the “values” implicit in AI actions are roughly chosen by the organisations who make them and by the customers who use them

I think this is the critical crux of the disagreement. A part of the Elizer’s argument, as I understand it, is that the current technology is completely incapable of anything close to actually “roughly choosing” the AI values. On this point, I think Elizer is completely right.

Anon User Apr 15, 2023, 8:19 PM
1 point
0
in reply to: Adam Golding’s comment on: Votes-per-Dollar
Hm? With the current system, at least the final vote counting process is relatively transparent. Yes, there are some opportunities to cheat on the margin of election finance laws, but importantly that opportunity is before the vote count, so has to be balances against the negative electoral consequences of being credibly accused of cheating. With you system, the final accounting happens after the vote, and in a close election, there is just too much incentive to cheat at that point...

Anon User Apr 15, 2023, 8:14 PM
1 point
0
on: GPT-4 is easily controlled/exploited with tricky decision theoretic dilemmas.
Interesting. But I am wondering—would the results been much different with pre-RLHF version of GPT-4? The GPT-4 paper has a figure showing that GPT-4 was close to perfectly calibrated before RLHF, and became badly calibrated after. Perhaps it’s something similar here?

Anon User Apr 10, 2023, 5:23 PM
3 points
0
on: Votes-per-Dollar
Voting has a dual role—not just determining the winner, but also demonstrating to the losers and their supporters that they lost fairly, in the most transparent way possible. How do you convince the losers that the winners did not cheat on their budget reporting? How do you account for “unpaid” volunteers? How do you account for “uncoordinated” spending by non-candidates?

Anon User Apr 8, 2023, 9:47 PM
3 points
0
on: Is it correct to frame alignment as “programming a good philosophy of meaning”?
Note that your “1” has two words that both carry very heavy load—“uses” and “correct”. What does it mean for a model to be correct? How do you create one? How do you ensure that the model you implemented in software is indeed correct? How do you create AI that actually uses that model under all circumstances? In patcicular, how do you ensure that it is stable under self-improvement, out-of-distribution environments, etc? Your “2-4″ seem to indicate that you are focusing more on the “correct” part, and not enough on the “uses” part. My understanding is that if both “correct” and “uses” could be solved, it would indeed likely be a solution to the alignment problem, but it’s probably not the only path, and not necessarily the most promising one. Other paths could potentially emerge from the work on AI corrigibility, negative side-effect minimization, etc.

Anon User Mar 26, 2023, 5:38 PM
1 point
0
on: How Politics interacts with AI ?

Rules will not support development of powerful AGI as it might threaten to overpower them

is probably true, but only because you used the word “powerful” rather than “capable”. Rulers would definitely want development of capable AGIs as long as they believe (however incorrectly) in their ability to maintain power/control over those AGIs.

In fact, rulers are likely to be particularly good at cultivating capable underlings they they maintain firm control of. It may cause them to overestimate their ability to do the same for AGI. In fact, if they expect an AGI to be less agentic, they might expect it to actually be easier to maintain control over an “we just program it to obey” AGI, and prefer that over what they perceive to be inherently less predictable humans.

Anon User Mar 25, 2023, 8:31 PM
11 points
7
on: Nudging Polarization
In modern politics, simple messages tend to work a lot better than nuanced ones (which is a thing that Donald Trump masterfully exploited). “X is good/bad” is a much simpler message than “X is good, but only if it’s X1, and not X2″ and having primary opponents claim “By supporting X, [politician] argees with the evil other-siders in their support for X2! [Politician] is an our-sider-in-name-only!”

Anon User Mar 25, 2023, 4:12 PM
3 points
1
in reply to: waterlubber’s comment on: Good News, Everyone!
Not just disinformation—any information that does not fit their preconceived worldview—it’s all “fake news”, don’t you know?

Anon User Mar 20, 2023, 4:46 PM
1 point
0
in reply to: avturchin’s comment on: Are COVID lab leak and market origin theories incompatible?
The lab is known to have been studying bats—weren’t those sold on the market too?

Anon User Mar 20, 2023, 4:44 PM
1 point
0
in reply to: noggin-scratcher’s comment on: Are COVID lab leak and market origin theories incompatible?

“Lab leak” doesn’t necessarily imply “created in a lab”.

Right, I was sloppy, replaced “created” with “studied”