Michaël Trazzi
Karma: 248
NewTop
Same issue here with the <a class=“users-name” href=“/users/mtrazzi”>Michaël Trazzi</a> tag. The e in “ë” is larger than the “a” (here is a picture).
The bug seems to come from font-family: warnock-pro,Palatino,“Palatino Linotype”,“Palatino LT STD”,“Book Antiqua”,Georgia,serif;” in .PostsPage-author (in <style data-jss=“” data-meta=“PostsPage”>).
If I delete this font-family line, the font changes but the “ë” (and any other letter with accent) appears to have the correct size.
Open Thread October 2018
You: Well.
The “You” should be bold.
typo: “Casual Decision Theory”
Let me see if I got it right:
-
Defining optimizers as an unpredictable process maximizing an objective function does not take into account algorithms that we can compute
-
Satisfying the property P “give the objective function higher values than an inexistence baseline” is not sufficient:
the lid satisfies (P) with “water quantity in bottle” but is just a rigid object that some optimizer put there. However, not the best counter-example because not a Yudkwoskian optimizer.
if a liver didn’t exist or did other random things then humans wouldn’t be alive and rich, so it satisfies (P) with “money in bank account” as the objective function. However, the better way to account for its behaviour (cf. Yudkowskian definition) is to see it as a sub-process of an income maximizer created by evolution.
One property that could work: have a step in the algorithm that provably augments the objective function (e.g. gradient ascent).
Properties I think are relevant:
intent: the lid did not “chose” to be there, humans did
doing something that the outer optimizer cannot do “as well” without using the same process as the inner optimizer : would be very tiring for humans to use our hands as lids. Humans cannot play go as well as Alpha Zero without actually running the algorithm.
-
it feels wrong to call other research dangerous, especially given its enormous potential for good.
I agree that calling 99.9% of AI research “dangerous” and AI Safety research “safe” is not an useful dichotomy. However, I consider AGI companies/labs and people focusing on implementing self-improving AI/code synthesis extremely dangerous. Same for any breakthrough in general AI, or things that greatly shorten the AGI timeline.
Do you mean that some AI research have positive expected utility (e.g. in medecine) and should not be called dangerous because the good they produce compensates for the increased AI-risk?
outside that bubble people still don’t know or have confused ideas about how it’s dangerous, even among the group of people weird enough to work on AGI instead of more academically respectable, narrow AI.
I agree. I run a local AI Safety Meetup and it’s frustrating to see that the ones who better understand the discussed concepts consider that Safety is way less interesting/important than AGI Capabilities research. I remember someone saying something like: “Ok, this Safety thing is kind of interesting, but who would be interested in working on real AGI problems” and the other guys noding. What they say:
“I’ll start an AGI research lab. When I feel we’re close enough to AGI I’ll consider Safety.“
”It’s difficult to do significant research on Safety without knowing a lot about AI in general.”
Bug: On Chrome using a Samsung Galaxy S7/Android 8.0.0 the “click and hold” thing does not work. Same with the “click to see how many people voted”.
Book Review: AI Safety and Security
Yes, typing mistakes in Turing Test is an example. It’s “artificially stupid” in the sense that you go from a perfect typing to a human imperfect typing. I guess what you mean by “smart” is an AGI that would creatively make those typing mistakes to deceive humans into believing it is human, instead of some hardcoded feature in a Turing contest.
The points we tried to make in this article were the following:
To pass the Turing Test, build chatbots, etc., AI designers make the AI artificially stupid to feel human-like. This tendency will only get worse as we get to interact more with AIs. The pb is that to have sth really “human-like” necessits Superintelligence, not AGI.
However, we can use this concept of “Artificial Stupidity” to limit the AI in different ways and make it human-compatible (hardware, software, cognitive biases, etc.). We can use several of those sub-human AGIs to design safer AGIs (as you said), or test them in some kind of sandbox environment.
If I understand you correctly, every AGI lab would need to agree in not pushing the hardware limits too much, even though they would steel be incentivized to do so to win some kind of economic competition.
I see it as a containment method for AI Safety testing (cf. last paragraph on the treacherous turn). If there is some kind of strong incentive to have access to a “powerful” safe-AGI very quickly, and labs decide to skip the Safety-testing part, then that is another problem.
Building Safer AGI by introducing Artificial Stupidity
Human-Aligned AI Summer School: A Summary
Added “AI” to prevent death from laughter.
I agree that the “Camp” in the title was confusing, so I changed it to “Summer School”. Thank you!
a treacherous turn involves the agent modeling the environment sufficiently well that it can predict the payoff of misbehaving before taking any overt actions.
I agree. To be able to make this prediction, it must already know about the preferences of the overseer, know that the overseer would punish unaligned behavior, potentially estimating the punishing reward or predicting the actions the overseer would take. To make this prediction it must therefore have some kind of knowledge about how overseers behave, what actions they are likely to punish. If this knowledge does not come from experience, it must come from somewhere else, maybe from reading books/articles/Wikipedia or oberving this behaviour somewhere else, but this is outside of what I can implement right now.
The Goertzel prediction is what is happening here.
Yes.
It’s important to start getting a grasp on how treacherous turns may work, and this demonstration helps; my disagreement is on how to label it.
I agree that this does not correctly illustrate a treacherous right now, but it is moving towards it.
Thanks for the suggestion!
Yes, it learned through Q-learning to behave differently when he had this more powerful weapon, thus undertaking multiple treacherous turn in training. A “continual learning setup” would be to have it face multiple adversaries/supervisors, so it could learn how to behave in such conditions. Eventually, it would generalize and understand that “when I face this kind of agent that punishes me, it’s better to wait capability gains before taking over”. I don’t know any ML algorithm that would allow such “generalization” though.
About an organic growth: I think that, using only vanilla RL, it would still learn to behave correctly until a certain threshold in capability, and then undertake a treacherous turn. So even with N different capability levels, there would still be 2 possibilities: 1) killing the overseer gives the highest expected reward 2) the aligned behavior gives the highest expected reward.
Congrats on your meditation! I remember commenting on your Prologue, about 80 days ago. Time flies!
Good luck with your ML journey. I did the 2011 Ng ML course, that uses Matlab, and Ng’s DL specialization. If you want to get a good grasp of recent ML I would recommend you to directly go to the DL specialization. Most of the original course is in the newer course, and the DL specialization uses more recent libraries (tf, keras, numpy).
This website is designed to make you write about three morning pages every day.
I’ve used it for about two years and wrote ~200k words.
Really recommend it to form an habit of daily free writing.