The only real answers at this point seem like mass public advocacy
If AI timelines are short, then I wouldn’t focus on public advocacy, but the decision-makers. Public opinion changes slowly and succeeding may even interfere with the ability of experts to make decisions.
I would also suggest that someone should focus on within-EA advocacy too (whilst being open about any possible limitations or uncertainties in their understanding).
You apparently completely misunderstood the point we were making with the white box thing.
I think you need to taboo the term white box and come up with a new term that will result in less confusion/less people talking past each other.
I appreciate how Ben handled this: it was nice for him to let me comment before he posted and for him to also add some words of appreciation at the end.
Regarding point 2, since I was viewing this in game mode I had no real reason to worry about being tricked. Avoiding being tricked by not posting about it would have been like avoiding losing in chess by never making the next move.
I guess other than that, I’d suggest that even a counterfactual donation of $100 to charity not occurring would feel more significant than the frontpage going down for a day. Like the current penalty feels like it was specifically chosen to be insignificant.
Also, I definitely would have taken it more seriously if I realised it was serious to people. This wasn’t even in my zone of possibility.
Sorry, I got tricked:
petrov_day_admin_account September 26, 2020 11:26 AM Hello Chris_Leong,
You are part of a smaller group of 30 users who has been selected for the second part of this experiment. In order for the website not to go down, at least 5 of these selected users must enter their codes within 30 minutes of receiving this message, and at least 20 of these users must enter their codes within 6 hours of receiving the message. To keep the site up, please enter your codes as soon as possible. You will be asked to complete a short survey afterwards.
I am very sorry to hear about your experiences. I hope you’ve found peace and that the organizations can take your experiences on board.
On one hand you seem to want there to be more open discussion around mental health, whilst on the other you are criticising MIRI and CFAR for having people have mental health issues in their orbit. These seem somewhat in tension with each other.
I think one of the factors is that the mission itself is stressful. For example, air traffic control and the police are high stress careers, yet we need both.
Another issue is that rationality is in some ways more welcoming of (at least some subset) people whom society would seem weird. Especially since certain conditions can be paired with great insight or drive. It seems like the less a community appreciates the silver-lining of mental health issues, the better they’d score according to your metric.
Regarding secrecy, I’d prefer for AI groups to lean too much on the side of maintaining precautions about info-hazards than too little. (I’m only referring to technical research not misbehaviour). I think it’s perfectly valid for donors to decide that they aren’t going to give money without transparency, but there should also be respect for people who are willing to trust/make the leap of faith.
It’s not the environment for everyone, but then again, the same could be said about the famously secretive Apple or say, working in national security. That seems like less of a organizational problem than one of personal fit (other alignment organizations and grant opportunities seem less secretive). One question: Would your position have changed it you felt that they had been more upfront about the stresses of the job?
Another point: I put the probability of us being screwed without Eliezer very high. This is completely different from putting him on a pedestal and pretending he’s perfect or that no-one can ever exceed his talents. Rather, my model is that without him it likely likely would have taken a number of more years for AI safety research to really kick off. And in this game even a few years can really matter. Now other people will have a more detailed model of the history than me and may come to different conclusions. But it’s not really a claim that’s very far out there. Of course, I don’t know who said it or the context so its hard for me to update on this.
Regarding people being a better philosopher than Kant or not, we have to take into account that we have access to much more information today. Basic scientific knowledge and understanding of computers resolves many of the disputes that philosophers used to argue over. Further, even if someone hasn’t read Kant directly, they may very well have read people who have read Kant. So I actually think that there would be a huge number of people floating around who would be better philosophers than Kant given these unfair advantages. Of course, it’s an entirely different question if we gave Kant access to today’s knowledge, but then again, given population growth, it wouldn’t surprise me if there were still a significant number of them.
One point I strongly agree with you on is that rationalists should pay more attention to philosophy. In fact, my most recent theory of logical counterfactuals was in part influenced by Kant—although I haven’t had time to read his critique of pure reason yet.
Look at the sidebar here? Is this anywhere near optimal? I don’t think so. Surely it should be encouraging people to undertake logical first steps towards becoming involved in alignment (ie. AGI safety fundamentals course, 80,000 hours coaching or booking a call with AI Safety Support).
In a few weeks, I’ll probably be spending a few hours setting up a website for AI Safety Australia and NZ (a prospective org to do local movement-building). Lots of people have web development capabilities, but you don’t even need that with things like Wordpress.
I’ve been spending time going through recent threads and encouraging people who’ve expressed interest in doing something about this, but are unsure what to do, to consider a few logical next steps.
Or maybe just reading about safety and answering questions on the Stampy Wiki (https://stampy.ai)?
Or failing everything else, just do some local EA movement building and make sure to run a few safety events.
I don’t know, it just seems like there’s low-hanging fruit all over the place. Not claiming these are huge impacts, but beats doing nothing.
What are you worried about CAIS doing?
I guess I’d be curious about your reasons of thinking that timelines are longer.
I don’t suppose you could clarify what the unresolved issues in decision theory are. What are the biggest issues that haven’t been solved for UDT or FDT? What is a co-ordination problem that hasn’t been solved? And what still isn’t known about counterfactuals?
“The structure of synchronization is, in general, richer than the world model itself. In this sense, LLMs learn more than a world model” given that I expect this is the statement that will catch a lot of people’s attention.
Just in case this claim caught anyone else’s attention, what they mean by this is that it contains:
• A model of the world
• A model of the agent’s process for updating its belief about which state the world is in
The author makes some good points, but:
I think they worry too much about the submissiveness of the relationship. I think it’s a much more common desire than people acknowledge. Not just in terms of sexuality, but in terms of a desire for a father figure or a great leader to tell people what to do. So it’s a common desire not just for particular moments, but in how people live their whole lives
I don’t agree with the point about relationship being invalid because you don’t have to work for it. I agree that this would be bad in a romantic relationship because it’d hamper your personal development, but I really don’t think that getting a cat instead of a dog will have a large effect. In fact, the safety provided by the unconditionality of a pets love may provide someone with the safety to take more risks in their relationships in the real world
I don’t think dogs need meaning in quite the same way as humans. They acknowledge that they’ve personified them to an extent and try to show their argument holds anyway. However, I don’t think they’ve entirely avoided the personification trap. I don’t deny that dogs may have instincts such as hunting or herding that are unfulfilled in modern life. This are instincts and we should be concerned with them being unfulfilled, but I don’t think we should equate them with a life purpose.
In any case, dogs as pets probably increase the empathy for animals significantly, so we should encourage more pets, not less.
I believe that the main reason why this hasn’t been discussed in any depth on Less Wrong is a) the norm set up by Elizier Yudkowsky in Politics is the Mindkiller b) The Motte has become the default rationalsphere adjacent location for this kind of discussion. That said, it’s plausible that the situation has reached the point where this topic can no longer be avoided.
This post clearly helped a lot of other people, but it follows a pattern that many other posts on Less Wrong also follow which I consider negative. The valuable contribution here is not the formalisation, but the generator behind the formalisation. The core idea appears to be the following:
“Human brains contain two forms of knowledge: - explicit knowledge and weights that are used in implicit knowledge (admittedly the former is hacked on top of the later, but that isn’t relevant here). Mary doesn’t gain any extra explicit knowledge from seeing blue, but her brain changes some of her implicit weights so that when a blue object activates in her vision a sub-neural network can connect this to the label “blue”.”
Unfortunately, there is a wall of maths that you have to wade through before this is explained to you. I feel it is much better when you provide your readers with a conceptual understanding of what is happening and only then include the formal details.
I’m still pretty skeptical of what would happen without explicit focus. The Bletchley Park declaration was a super vague and applause-lighty declaration, which fortunately mentions issues of control, but just barely. It’s not clear to me yet that this will end up receiving much-dedicated focus.
Regarding biosecurity and cyber, my big worry here is open-source and it seems totally plausible that a government will pass mostly sensible regulation, then create a massive gaping hole where open-source regulation should be.
I’m much more skeptical about liability.
My worry is that these lawsuits would become the overriding factor in terms of what decisions are made about safety to the point where other considerations, such as what is actually safe, will be driven out.
Just as a comment, the Stampy Wiki is also trying to do the same thing, but it’s a good idea as it’s more convenient for many people to ask on Less Wrong.
I would encourage you and other commentators to read Singer’s article Poverty, Affluence and Morality in which he develops his argument in more detail. It’s an extremely easy read and not overly long either (less than 20 pages). He explicitly adresses the potentially overwhelming nature of our obligations. He also addresses the issue of overpopulation that Vanilla Cabs raises. If I had more time, I’d provide a summary of what he says, but unfortunately I’ll have to leave this for someone else.
So it’s worth understanding that this analogy is where his argument begins and not where it ends. I don’t think he would claim that you should make your decision upon the basis of this thought experiment by itself.
I’ve attended a CFAR workshop. I agree with you that Double Crux has all of these theoretical flaws, but it actually seems to work reasonably well in practise, even if these flaws make it kind of confusing. In practise you just kind of stumble through. I strongly agree that if the technique was rewritten so that it didn’t have these flaws, it would be much easier to learn as the stumbling bit isn’t the most confidence inspiring (this is when the in person assistance becomes important).
One of the key elements that haven’t seen mentioned here is this separation between trying to persuade the other person and trying to find out where your point of view differs. When you are trying to convince the other person it is much easier to miss, for example, when there’s a difference in a core assumption. Double Crux lets you understand the broad structure of their beliefs so that you can at least figure out the right kinds of things to say later to persuade them that won’t be immediately dismissed.
Hey Connor, I’m really happy to see someone else trying to help extract the value from Dominic Cummings work. Here’s a link to a summary of wrote of his ideas.
So I’m confused—where can we access the wiki now? You said some were classed as tags and others weren’t? So are *all* the articles under concepts; or only some? And how do we create a new wiki page?