Hi I am working on Rob Miles’ Stampy project (https://aisafety.info/), which is creating a centralized resource for answering questions about AI safety and alignment. Would we be able to incorporate your list of frequently asked questions and answers into our system (perhaps with some modification)? I think they are really nice answers to some of the basic questions and would be useful for people curious about the topic to see.
Yaakov T
Karma: 233
have you seen the stampy project https://aisafety.info/ although it is currently a work in progress. there was also some examples of it here https://www.lesswrong.com/posts/EELddDmBknLyjwgbu/stampy-s-ai-safety-info-new-distillations-2
@drocta @Cookiecarver We started writing up an answer to this question for Stampy. If you have any suggestions to make it better I would really appreciate it. Are there important factors we are leaving out? Something that sounds off? We would be happy for any feedback you have either here or on the document itself https://docs.google.com/document/d/1tbubYvI0CJ1M8ude-tEouI4mzEI5NOVrGvFlMboRUaw/edit#
But in that kind of situation, wouldn’t those people also pick A over B for the same reason?
I really liked this post since it took something I did intuitively and haphazardly and gave it a handle by providing the terms to start practicing it intentionally. This had at least two benefits:
First it allowed me to use this technique in a much wider set of circumstances, and to improve the voices that I already have. Identifying the phenomenon allowed it to move from a knack which showed up by luck, to a skill.
Second, it allowed me to communicate the experience more easily to others, and open the possibility for them to use it as well. Unlike many lesswrong posts, I found that the technique in this post spoke to a bunch of people outside of the lesswrong community. For example, one friend who liked this idea. tried applying it to developing an Elijah the Prophet figure that he could interact with.
Cool. So in principle we could just as well use the rationals from the standpoint of scientific inference. But we use the reals because it makes the math easier. Thank you.
Thank you.
I am a little confused. I was working with a definition of continuity mentioned here https://mathworld.wolfram.com/RationalNumber.html : “It is always possible to find another rational number between any two members of the set of rationals. Therefore, rather counterintuitively, the rational numbers are a continuous set, but at the same time countable.”
I understand that Rationals aren’t complete, and my question is why this is important for scientific inference. In other words, are we using the reals only because it makes the math easier, or is there a concrete example of inference which completeness helps with?
Specifically, since the context Jaynes is interested in is designing a (hypothetical) robot’s brain, and in order to achieve that we need to associate degrees of plausibility with a physical state, I don’t see why that entails the property of completeness which you mentioned? In fact, we mostly use digital and not analog computers, which use rational approximations for the reals. What does this system of reasoning lack?
You might find the book Is Water H2O? by Hasok Chang, 2012 useful. It was mentioned by Adam Shimi in this post https://www.lesswrong.com/posts/wi3upQibefMcFs5to/levels-of-pluralism
It also reminds me of Richard Feynman not wanting a position at the institute for advance study.
“I don’t believe I can really do without teaching. The reason is, I have to have something so that when I don’t have any ideas and I’m not getting anywhere I can say to myself, “At least I’m living; at least I’m doing something; I am making some contribution”—it’s just psychological.
When I was at Princeton in the 1940s I could see what happened to those great minds at the Institute for Advanced Study, who had been specially selected for their tremendous brains and were now given this opportunity to sit in this lovely house by the woods there, with no classes to teach, with no obligations whatsoever. These poor bastards could now sit and think clearly all by themselves, OK? So they don’t get any ideas for a while: They have every opportunity to do something, and they are not getting any ideas. I believe that in a situation like this a kind of guilt or depression worms inside of you, and you begin to worry about not getting any ideas. And nothing happens. Still no ideas come.
Nothing happens because there’s not enough real activity and challenge: You’re not in contact with the experimental guys. You don’t have to think how to answer questions from the students. Nothing!
In any thinking process there are moments when everything is going good and you’ve got wonderful ideas. Teaching is an interruption, and so it’s the greatest pain in the neck in the world. And then there are the longer period of time when not much is coming to you. You’re not getting any ideas, and if you’re doing nothing at all, it drives you nuts! You can’t even say “I’m teaching my class.”
If you’re teaching a class, you can think about the elementary things that you know very well. These things are kind of fun and delightful. It doesn’t do any harm to think them over again. Is there a better way to present them? The elementary things are easy to think about; if you can’t think of a new thought, no harm done; what you thought about it before is good enough for the class. If you do think of something new, you’re rather pleased that you have a new way of looking at it.
The questions of the students are often the source of new research. They often ask profound questions that I’ve thought about at times and then given up on, so to speak, for a while. It wouldn’t do me any harm to think about them again and see if I can go any further now. The students may not be able to see the thing I want to answer, or the subtleties I want to think about, but they remind me of a problem by asking questions in the neighborhood of that problem. It’s not so easy to remind yourself of these things.
So I find that teaching and the students keep life going, and I would never accept any position in which somebody has invented a happy situation for me where I don’t have to teach. Never.”
— Richard Feynman, Surely You’re Joking, Mr. Feynman!
Do you suspect that black-box knowledge will be transferable between different models, or that the findings will be idiosyncratic to each system?
according to this website (http://ravallirepublic.com/news/opinion/viewpoint/article_876e97ba-1aff-11e2-9a10-0019bb2963f4.html) it is part of ‘aphorisms for leo baeck’ (which I think is printed in ‘ideas and opinions’ but I don’t have access to the book right now to check)
probably not, but why are you certain
That doesn’t strike me as how psychology works, since in the real world people often repeatedly make the same mistakes. It also seems that even if your proposal would work, it doesn’t address the original issue since you are assuming that the person has a clear idea of his goals and only needs time to pursue them, whereas I think the bigger issue which aging encourages is reorienting ones values.
I appreciate your taking the time to address my question, but it seems to me that this conversation isn’t really making progress so I will probably not respond to future comments on this thread. Thank you
I would have to look around to see if there is non-anecdotal evidence, but anecdotally ~40 is when I have heard people start mentioning it.
I don’t think your proposal would work since I don’t think the time factor is the biggest issue, How often do people make big plans for summer vacation and not actually do them? They probably wouldn’t say “I’ll put it off for thirty years”, but rather repeatedly say ” I’ll put it off till tomorrow” .
yes and that was the meaning of my initial comment, and that is a concern in today’s world where we do have limited resources so that not everyone would be able to make use of such a technology. The country that has it (or the subset of people that have it within one country) will be motivated to defend their resources necessary to use it., This isn’t an argument against such research in a world without any scarcity, but that isn’t our world.
I am still not sure whether it is likely to be more beneficial or not for heavily emotional and biased humans like us.
Thank you for all of your clarifications, I think I now understand how you are viewing morality.
Maybe, but on the other hand there is inequity aversion: http://en.wikipedia.org/wiki/Inequity_aversion
Also there is the possibility of fighting over the resources to use that technology (either within society or without). Do you disagree with the general idea that without greater rationality extreme longevity will not necessarily be beneficial or do you only disagree with the example?
Why don’t you view the consequentialist imperative to always seek maximum utility as a deontological rule? If it isn’t deontological where does it come from?
“You keep using the words “we” and “our”, but “we” don’t have lifespans; individual humans do.” Of course, but “we” is common shorthand for decisions which are made at the level of society, even though that is a collection of individual decisions (e.g. should we build a bridge, or should we legalize marijuana). Do you think that using standard english expressions is problematic? (I agree that both the question of benefit for the self and benefit for others is important and think the issue of cognitive biases is relevant to both of them)
I just looked at your comment, and I agree with that argument, but that hasn’t been my impression of the view of many on this site (and clearly isn’t the view of researchers like De Grey), however I am relatively new here and may be mistaken about that. Thank you for clarifying.
https://en.wikipedia.org/wiki/Von_Neumann%E2%80%93Morgenstern_utility_theorem
https://www.lesswrong.com/tag/vnm-theorem