Isn’t it a problem if the people most concerned with alignment refuse to participate in AI development? Aside from top-down corporate safety inititives, It seems like that would mean that the entire field is selecting for people who are unconcerned with alignment.
blackstampede
Karma: 48
I’m lumping them together because they could (potentially) increase the likelihood of advances in AI. If some number of people read books about deep learning, then it’s likely that some fraction of those people will go on to contribute in some small way to the field. Educating people about AI, publishing papers on AI, criticizing papers on AI in a constructive way- even using open source AI platforms could increase demand for more and better AI products.
I don’t entirely buy this, but someone could argue that anything related to the study of ML/AI is dangerous (see my conversation with Trevor1 down-thread for an immediate example).
I wouldn’t call it an infohazard; generally that refers to information that’s harmful simply to know, rather than because it might e.g. advance timelines.
Ah ok. That was my understanding as well, but I’ve seen infohazard used to refer to things where a wider awareness could be bad.
There are arguments to be made about how much overlap there is between capabilities research and alignment research, but I think by default most things that would be classified as capabilities research do not meaningful advance AI alignment.
If this is true, is there not still value in alignment researchers contributing to capabilities advancement in order to be in the room when it’s made?
I agree that studying AI and trying to develop AI are two different things. I think you can lump them together for this conversation- just exclude “working on alignment” from the cluster and everything else (studying AI, developing AI, improving AI) seems to be considered an infohazard that should be avoided.
I don’t think I agree that advancing AI capabilities is definitely bad. It shortens timelines, which is bad. But it seems to me that figuring out AI alignment would take a lot of the same research work that advancing AI capabilities would take. Refusing to participate in the advancement of AI capabilities handicaps alignment research and leaves the cutting edge work to groups who don’t care about alignment.
In the first paragraph, I think you’re saying that discussing AGI/AI on the internet is hazardous because media organizations are incentivized to vilify you. After that, you lost me.
Are you saying that people who are refusing to study AI should be prevented from leaving the field (or encouraged to stay some how)? Can you expand on this?
[Question] Is the study of AI an infohazard?
I forgot where I saw this, but there’s a strategy where the person betting that humanity will survive longer than a year gives the person betting on doom the money in advance, with the doomer returning the money plus interest if they’re wrong. I forgot the details. But if you’re looking for a way to bet on the end of the world, it’s the only way I can think of.
EDIT: I think I saw this in a post about EY betting with someone.
No worries, I appreciate the discussion.
I use terms like nation, state and country because that’s what I’ve grown up with, but you could replace all of them with “system for organizing group action” and I think the argument would be the same. We have bad systems of organization and we should consider improving them. The improvements may not be useful to us but they may be useful to someone in the future, and could potentially improve a lot of lives. That feels like a goal that longtermists would support. That’s what I was trying to say in a nutshell- if a Walkaway-style anarchist society is better and freer for most people, then I think we should pursue that. If it’s not, then we shouldn’t.
Abortion law can be approached from whatever angle you like, but most people would still consider it to be related to their morality- either because it impacts bodily autonomy or life (or both), depending on what you think of it. Whether a legislative group made their decision based on legitimate amoral concerns isn’t going to affect whether it is a moral question or not because that’s a judgement that is made at the individual level.
I feel like I’m not explaining this as well as I should be able to. But it feels like both things could be true. The law could be as it is because it promotes justice or equality or any number of other broadly good concepts that the legislators support for the good of a society, and at the same time the law could be (in some fundamental way) a moral judgement just because individuals view it as such.
I’m vegetarian. I think eating meat is unethical. I think it’s unethical for myself as well as others, although I’m more concerned with my own ethics than I am with the ethics of others. If the government, through a purely logical, data-driven process, decided that outlawing the slaughter of animals and the sale of meat in the US was best for the country, I would consider that to be a morally good law. It would seem like a judgement in favor of my morality and against the morality of a person who eats meat. If others felt the same way, then the government is, in effect, making a determination on a moral question.
If that makes sense.
In my (probably naive) view, the government should provide a stable, minimal (?), legal structure that individuals can exercise their own moral judgements within. Sometimes the government has to legislate on moral questions but that should be limited.
Abortion comes to mind as an example of a moral question that the government has to legislate on. An example of a framework for individuals to work within might be marriage- the government should provide contract law, but the the religious and moral aspects of marriages are provided by the participants.
(I know that marriage in the US isn’t currently treated this way, but I think it probably should be)
I agree that they were trying to make a stable political system that follows this sort of defense-in-depth pattern (I used the US as an example of this in an edit) and depending on what your criteria for success is, they succeeded. But my gut-feeling is that the US wont last another 250 − 500 years without either dissolving completely or falling into some failure-mode that leads to significantly less freedom. I think that, given what we know today, we could probably design a better system. One example that seems like a pretty obvious improvement over the current US system is ranked-choice voting.
EDIT: I just realized that I didn’t really respond to the quote you used. The “convoluted, haphazard and error-prone” comment was generally referring to the modifications and traditions that have grown up over the original system. I agree that the original government was an attempt at a defense-in-depth pattern.
Thanks- “well thought out” is generous, but maybe I can edit/continue to expand on it and make something a little more solid out of it.
Making stable, free nations as a hobby
Yeah, that seemed like a given to me as well. I’ve already built much of my project around a local user-specific-posts-and-replies and a broader all-users-and-replies feed.
Ok, thanks. I’m keeping a list of interesting ideas, and I’ve added this.
Thanks- I hadn’t thought of that and didn’t realize you could do it. What is the syntax for that on LessWrong?
I’m not a published academic and I haven’t done any serious analysis to validate this, but I think that improving transparency of academic contribution might provide motivation for peer review. The hard work of evaluating and filtering published research would be attractive if it were publicly recognized. If a reputation could be built around critically analyzing and responding to new research rather than just publishing, then more people would do it, whether it was paid or not.
If participation yields some amount of insight on alignment then it’s not clear to me that no participation is obviously the better course of action over participation. To argue that, I think that someone would have to show that working on (and learning about) AI is of no (or little) value to the study of alignment. It seems possible that a shorter timeline could be worth it if the development process also accelerates alignment work.
At the very least, I haven’t seen this tradeoff addressed in a serious way. If it has, I’d appreciate a link.