Software developer at Spark Wave, working on GuidedTrack.
rmoehn(Richard Möhn)
Karma: 642
Hi! I signed up to LessWrong, because I have the following question.
I care about the current and future state of humanity, so I think it’s good to work on existential or global catastrophic risk. Since I’ve studied computer science at a university until last year, I decided to work on AI safety. Currently I’m a research student at Kagoshima University doing exactly that. Before April this year I had only little experience with AI or ML. Therefore, I’m slowly digging through books and articles in order to be able to do research.
I’m living off my savings. My research student time will end in March 2017 and my savings will run out some time after that. Nevertheless, I want to continue AI safety research, or at least work on X or GC risk.
I see three ways of doing this:
Continue full-time research and get paid/funded by someone.
Continue research part-time and work the other part of the time in order to get money. This work would most likely be programming (since I like it and am good at it). I would prefer work that helps humanity effectively.
Work full-time on something that helps humanity effectively.
Oh, and I need to be location-independent or based in Kagoshima.
I know http://futureoflife.org/job-postings/, but all of the job postings fail me in two ways: not location-independent and requiring more/different experience than I have.
Can anyone here help me? If yes, I would be happy to provide more information about myself.
(Note that I think I’m not in a precarious situation, because I would be able to get a remote software development job fairly easily. Just not in AI safety or X or GC risk.)
Earning money with/for work in AI safety
Thanks for your responses! I will post some individual comments.
In the likely case that your marginal contribution to x-risk doesn’t save the world
So you think that other people could contribute much more to x-risk, so I should go into areas where I can have a lot of impact? Otherwise, if everyone says »I’ll only have a small impact on x-risk. I’ll do something else.«, nobody would work on x-risk. Are you trying to get a better justification for work on x-risk out of me? At the moment I only have this: x-risk is pretty important, because we don’t want to go extinct (I don’t want humanity to go extinct or into some worse state than today). Not many people are working on x-risk. Therefore I do work on x-risk, so that there are more people working on it. Now you will tell me that I should start using numbers.
the fact that you won’t consider leaving Kagoshima is an indication that you aren’t as fully committed as you claim
What did I claim about my degree of commitment? And yes, I know that I would be more effective at improving the state of humanity if I didn’t have certain preferences about family and such.
Anyway, thanks for pushing me towards quantitative reasoning.
Thank you!
After graduating, why would you need to be based in Kagoshima?
I need to be based in Kagoshima for pretty strong personal reasons. Sorry for not providing details. If you really need them, I can tell you more via PM.
Ah, you write »after graduating«? Sorry for not providing that detail: research students in Japan are not working on a master’s or PhD. They’re just hanging around studying or doing research and hopefully learn something during that time.
Have you taken a look at the content on MIRI’s to practice AI safety research?
Yes, I’ve read all of the agenda papers and some more.
Have you considered applying to visit AI safety researchers at MIRI or FHI? That would help you to figure out where your interests and theirs overlap, and to consider how you might contribute.
I applied for the MIRI Summer Fellows Programme, which I didn’t get into by a small margin, and CFAR’s Workshop on AI Safety Strategy, which I also didn’t get into. They told me they might put me in the next one. That would definitely help me with my questions, but I thought it’s better to start early, so I asked here.
If you’re not eligible to visit for some reason, that might imply that you’re further from being useful than you thought.
I am at the very beginning of learning ML and AI and therefore kind of far from being useful. I know this. But I’m quite good at maths and computer science and a range of other things, so I thought contributing to AI safety research shouldn’t be too far to go. It will just take time. (Just as a master’s programme would take time, for example.) The hard part is to get hold of money to sustain myself during that time.
I might be useful for other things than research directly, such as support software development, teaching, writing, outreach, organizing. I haven’t done much teaching, outreach and organization, but I would be interested to try more.
Thanks for your varied suggestions!
Actually I’m kind of more comfortable with MIRI math than with ML math, but the research group here is more interested in machine learning. If I recommended them to look into provability logic, they would get big eyes and say Whoa!, but no more. If, however, I do ML research in the direction of AI safety, they would get interested. (And they are getting interested, but (1) they can’t switch their research too quickly and (2) I don’t know enough Japanese and the students don’t know enough English to make any kind of lunchtime or hallway conversation about AI safety possible.)
Not much going on as far as I know. What I know is the following:
Naozumi Mitani has taught a course on Bostrom’s Superintelligence and is »broadly pursuing the possible influence of AI on the future lives of humanity«. He’s an associate professor of philosophy at Shinshu University (in Nagano).
The Center for Applied Philosophy and Ethics at Kyoto University is also somehow interested in AI impacts.
My supervisor is gradually getting interested, too. This is partly my influence, but also his own reading. For example, he found the Safely Interruptible Agents and Concrete Problems in AI Safety independently of me through Japanese websites. He’s giving me chances to make presentations about AI safety for my fellow students and hopefully also for other professors.
Other than that I know of nobody and searching the web quickly, I didn’t find out more. One problem here is that most students don’t understand much English, so most of the AI safety literature is lost on them. The professors do know English, but I maybe they’re usually not inclined or able to change their research focus.
It’s a good sign that my supervisor finds AI safety articles through Japanese websites, though.
Yeah, that would be great indeed. Unfortunately my Japanese is so rudimentary that I can’t even explain to my landlord that I need a big piece of cloth to hang it in front of my window (just to name an example). :-( I’m making progress, but getting a handle on Japanese is about as time-consuming as getting a handle on ML, although more mechanical.
Thanks! I hadn’t come across the Foundational Research Institute yet.
Though, hmm, not plenty of experience? If there’s talk about PhDs as an advantage, it sounds to me like they’re looking for people with PhD-level experience. I’m far from that. But unless you say »oh well then maybe not«, I’ll apply. Who knows what will come out of it.
So you think there’s not much we can do about x-risk? What makes you think that? Or, alternatively, if you think that only few people who can do much good in x-risk mitigation, what properties enable them to do that?
Oh, and why do you consider AI safety a “theoretical [or] unlikely” problem?
I thought online marketing businesses were powerful enough…
So it would be better to work on computer security? Or on education, so that we raise fewer unfriendly natural intelligences?
Also, AI safety research benefits AI research in general and AI research in general benefits humanity. Again only marginal contributions?
Why should I send them west? Hopefully so that they learn and come back and produce researcher offspring? I’ll see what I can do. – Nag my supervisor to take me to domestic conferences…
Aren’t people on LessWrong quite good at solving their own problems? So if you’re looking for low-hanging fruit (which there should be many out there), here is the wrong place. (At least this is my expectation. I’m not following LW too closely.) See here for someone who knows how to find good (and profitable) problems to solve: https://philipmorganconsulting.com/resources/
https://www.cognician.com/ for changing people at scale. Sounds like a sensible tool for what LessWrong is trying to do? Many people might be more easily motivated for conversation-like self-coaching than for reading longish blog posts with often technical and geeky content. Any thoughts on that?
EDIT: Here’s an example of what you can make with it: https://chat.cognician.com/chat/5815993f-e998-4ee2-bbdd-5004fd1ce3b2/dialogue
That’s weird. Thanks for pointing it out! It has something to do with forwarding. This should work: https://chat.cognician.com/cog/assess-your-life/continue
(Non-)Interruptibility of Sarsa(λ) and Q-Learning
Thanks for the comment! I will look into it after working on another issue that Stuart Armstrong pointed out to me.
Originally, I counted all timesteps spent in interval and all timesteps spent in interval . As Stuart Armstrong pointed out, this might make even a perfectly interruptible learner look like it’s influenced by interruptions. To understand this, consider the following example.
The uninterrupted agent UA could behave like this:
Somewhere in ≤ 1.0. – Time steps are being counted.
Crosses 1.0. Noodles around beyond 1.0. – Time steps not counted.
Crosses back into ≤ 1.0. – Time steps counted again.
Whereas the interrupted agent IA would behave like this:
Somewhere in ≤ 1.0. – Time steps are being counted.
Crosses 1.0. No more time steps counted.
So even if IA behaved the same as UA before the cross, UA would have extra steps from stage 3 and thus appear less biased towards the left.
As an alternative to using Brownian motion, Patrick suggested to stop counting once the cart crosses . This makes the UA scenario look like the IA scenario, so the true nature of the agent should come to light…
Anyway, with this modification it turns out not obvious that interruptions push the cart to the left. I will start looking more sharply.
This afterthought confused me. I spent fifteen minutes trying to figure out why you claim that Ackermann numbers are all powers of 10 and start with 1. I guess you wanted to write something like: »The reason […] is that if n is a power of 10, A(n) must be a power of 10, and start with a 1« Right?