Michaël Trazzi
Karma: 254
NewTop
(the account databases are shared, so every LW user can log in on alignment forum, but it will say “not a member” in the top right corner)
I am having some issues in trying to log in from a github-linked account. It redirects me to LW with an empty page and does nothing.
This website is designed to make you write about three morning pages every day.
I’ve used it for about two years and wrote ~200k words.
Really recommend it to form an habit of daily free writing.
Same issue here with the <a class=“users-name” href=“/users/mtrazzi”>Michaël Trazzi</a> tag. The e in “ë” is larger than the “a” (here is a picture).
The bug seems to come from font-family: warnock-pro,Palatino,“Palatino Linotype”,“Palatino LT STD”,“Book Antiqua”,Georgia,serif;” in .PostsPage-author (in <style data-jss=“” data-meta=“PostsPage”>).
If I delete this font-family line, the font changes but the “ë” (and any other letter with accent) appears to have the correct size.
Open Thread October 2018
You: Well.
The “You” should be bold.
typo: “Casual Decision Theory”
Let me see if I got it right:
-
Defining optimizers as an unpredictable process maximizing an objective function does not take into account algorithms that we can compute
-
Satisfying the property P “give the objective function higher values than an inexistence baseline” is not sufficient:
the lid satisfies (P) with “water quantity in bottle” but is just a rigid object that some optimizer put there. However, not the best counter-example because not a Yudkwoskian optimizer.
if a liver didn’t exist or did other random things then humans wouldn’t be alive and rich, so it satisfies (P) with “money in bank account” as the objective function. However, the better way to account for its behaviour (cf. Yudkowskian definition) is to see it as a sub-process of an income maximizer created by evolution.
One property that could work: have a step in the algorithm that provably augments the objective function (e.g. gradient ascent).
Properties I think are relevant:
intent: the lid did not “chose” to be there, humans did
doing something that the outer optimizer cannot do “as well” without using the same process as the inner optimizer : would be very tiring for humans to use our hands as lids. Humans cannot play go as well as Alpha Zero without actually running the algorithm.
-
it feels wrong to call other research dangerous, especially given its enormous potential for good.
I agree that calling 99.9% of AI research “dangerous” and AI Safety research “safe” is not an useful dichotomy. However, I consider AGI companies/labs and people focusing on implementing self-improving AI/code synthesis extremely dangerous. Same for any breakthrough in general AI, or things that greatly shorten the AGI timeline.
Do you mean that some AI research have positive expected utility (e.g. in medecine) and should not be called dangerous because the good they produce compensates for the increased AI-risk?
outside that bubble people still don’t know or have confused ideas about how it’s dangerous, even among the group of people weird enough to work on AGI instead of more academically respectable, narrow AI.
I agree. I run a local AI Safety Meetup and it’s frustrating to see that the ones who better understand the discussed concepts consider that Safety is way less interesting/important than AGI Capabilities research. I remember someone saying something like: “Ok, this Safety thing is kind of interesting, but who would be interested in working on real AGI problems” and the other guys noding. What they say:
“I’ll start an AGI research lab. When I feel we’re close enough to AGI I’ll consider Safety.”
“It’s difficult to do significant research on Safety without knowing a lot about AI in general.”
Bug: On Chrome using a Samsung Galaxy S7/Android 8.0.0 the “click and hold” thing does not work. Same with the “click to see how many people voted”.
Book Review: AI Safety and Security
Yes, typing mistakes in Turing Test is an example. It’s “artificially stupid” in the sense that you go from a perfect typing to a human imperfect typing. I guess what you mean by “smart” is an AGI that would creatively make those typing mistakes to deceive humans into believing it is human, instead of some hardcoded feature in a Turing contest.
The points we tried to make in this article were the following:
To pass the Turing Test, build chatbots, etc., AI designers make the AI artificially stupid to feel human-like. This tendency will only get worse as we get to interact more with AIs. The pb is that to have sth really “human-like” necessits Superintelligence, not AGI.
However, we can use this concept of “Artificial Stupidity” to limit the AI in different ways and make it human-compatible (hardware, software, cognitive biases, etc.). We can use several of those sub-human AGIs to design safer AGIs (as you said), or test them in some kind of sandbox environment.
If I understand you correctly, every AGI lab would need to agree in not pushing the hardware limits too much, even though they would steel be incentivized to do so to win some kind of economic competition.
I see it as a containment method for AI Safety testing (cf. last paragraph on the treacherous turn). If there is some kind of strong incentive to have access to a “powerful” safe-AGI very quickly, and labs decide to skip the Safety-testing part, then that is another problem.
Building Safer AGI by introducing Artificial Stupidity
Human-Aligned AI Summer School: A Summary
Added “AI” to prevent death from laughter.
I agree that the “Camp” in the title was confusing, so I changed it to “Summer School”. Thank you!
a treacherous turn involves the agent modeling the environment sufficiently well that it can predict the payoff of misbehaving before taking any overt actions.
I agree. To be able to make this prediction, it must already know about the preferences of the overseer, know that the overseer would punish unaligned behavior, potentially estimating the punishing reward or predicting the actions the overseer would take. To make this prediction it must therefore have some kind of knowledge about how overseers behave, what actions they are likely to punish. If this knowledge does not come from experience, it must come from somewhere else, maybe from reading books/articles/Wikipedia or oberving this behaviour somewhere else, but this is outside of what I can implement right now.
The Goertzel prediction is what is happening here.
Yes.
It’s important to start getting a grasp on how treacherous turns may work, and this demonstration helps; my disagreement is on how to label it.
I agree that this does not correctly illustrate a treacherous right now, but it is moving towards it.
Thanks for the post!
It resonates with some experience I had in playing the game of go at a competitive level.
Go is a perfect information game but it’s very hard to know exactly what will be the outcome of a “fight” (you would need to look up to 30 moves ahead in some cases).
So when the other guy would kill your group of stones after a “life or death” scenario, because he had a slight advantage in the fight, it feels like the other is lucky, and most people have really bad thoughts and just give up.
Once, I created an account with the bio “I don’t resign” to see what would happen if I forced myself to not concede and keep playing after a big loss. It went surprisingly well and I even went to play the highest ranked guy connected on the server. At this point, I completely lost the game and there was 100+ people watching the game, so I just resigned.
Looking back, it definitely helped me to continue fighting even after a big loss, and stop the mental chatter. However, there’s a trade-off between the time gained by correctly estimating the probability of winning and resigning when too improbable, and the mental energy gained from not resigning (minus the fact that your opponent may be pretty pissed off).