If you are going to read just one thing I wrote, read The Problem of the Criterion.
More AI related stuff collected over at PAISRI
G Gordon Worley III(G Gordon Worley III)
Karma: 6,618
It doesn’t seem that way to me. As best I can tell Arbital failed at its own goals because we’re not using it as a repository for knowledge about AI alignment, so by its own objectives it was not working fine.
I dread when I have to look something up in Arbital. It’s slow, which is fixable, but also somewhat confusing to navigate. I actually agree with Tofly’s sentiment that I think from the outside it seems like Arbital tried to bite off more than it could chew rather than trying to address problems incrementally.
Can confirm. I almost took a job with the federal government early in my career but passed because I could get paid more with less bullshit elsewhere (e.g. no strict dress code).
+1, I think this is a great perspective. Everything I’ve written on Less Wrong was because I decided it was better to work full time in software engineering and do academic-like stuff on the side rather than the other way around. As a result I did a bunch of work on AI safety among other things (whether it amounts to much is a separate question). I think this is a pretty important and viable career path, and I’m sort of surprised I hadn’t seen a post calling it out explicitly before, especially not one written by me since I’ve been doing the same for so long!
I haven’t heard of CTMU.
I generally score INTP on MBTI.
This seems cool but I think I’m missing, after reading it, something like the pattern of what it would mean to interact with it other than just share a comment with a list of patterns, which feels like a dead action of sending information into the void.
Re Project 4, you might find my semi-abandoned (mostly because I wasn’t and still am not in a position to make further progress on it) research agenda for deconfusing human values useful.
Re: Project 2
This project’s goal is to better understand the bridge principles needed between subjective, first person optimality and objective, third person success.
This seems quite valuable, because there is, properly speaking, no objective, third person perspective on which we can speak, only the inferred sense that there exists something that looks to use like a third person perspective from our first person perspectives. Thus I think this seems like a potentially fruitful line of research since the proposed premise contains the confusion that needs to be unraveled to get to addressing what is something more like the intersubjective agreement on what the world is like.
In a world that is truly and completely post-scarcity there would be no need for making tradeoffs.
Normally when we think about a post-scarcity future we think in terms of physical resources like minerals and food and real estate because for many people these are the limiting resources.
But the world is wealthy enough that some people already have access to this kind of post-scarcity. That is, they have enough money that they are not effectively limited in access to physical resources. If they need food, shelter, clothing, materiel, etc. they can get it in sufficient quantities to satisfy their needs. And yet these post-scarcity people still have a scarce resource they can’t get enough of: time.
Because time is limited, we must be judicious in its use. In a world of true post-scarcity, there would be enough time that it would be effectively unlimited, say quadrillions of years of subjective experience. With so much time on our hands, we would not need to make judicious use of time. And that goes not just for our own time but for the time of others.
Today much of how we pick jobs, romantic partners, hobbies, etc. is determined by tradeoffs. Given many options but limited time and resources, we pick ones that let us maximize given the constraints. For example, maybe I’d really like to teach history and paint landscapes, but teaching history doesn’t pay me much and painting landscapes takes more spare time than I have, so instead I get a job in accounting and watch TikToks. They offer me a better tradeoff in terms of the things I want: getting paid and having fun with my spare time. I’m not necessarily happy that I cannot do better, but I am happy to at least do the best I can.
In a world with effectively unlimited time, these constraints would not apply and there would be no need to make tradeoffs.
Now, unfortunately, I don’t expect to ever see such a true world of post-scarcity. There’s a few things that are absolutely limited in this world like time and energy. Although we can get a lot of them, perhaps more than we know what to do with, they’re still limited, and so long as there’s sufficient competition from others I expect those resources to get eaten up. Lots of free time and energy? Let’s create some more people to make use of them. Maybe not Malthusian Trap, repugnant conclusion many of them, but a lot of them. Enough that we have to make time tradeoffs again. If we’re not making time tradeoffs, we’re not making enough people or using enough energy.
And so I expect to always face these limits. We’ll likely face different limits than we do today. For example, I’m not worried that in a post-scarcity future we wouldn’t have time to both teach history and paint landscapes if that’s what we wanted to do, but I do worry that we’d only have time to spend 10^29th lifetimes as a bird or snail, never getting to fully explore the full space of possible experiences. It’ll be a different kind of tradeoff we’ll have to make, but a tradeoff nonetheless.
Oh don’t worry, I’m going to argue in a later chapter that there are no mind independent facts at all. 😊
Fundamental Uncertainty: Chapter 3 - Why can’t we all agree on what’s right?
Given the stakes are so high here and that there’s potential for preventing this from getting out of hand, I’d recommend immediate and total containment. Make it clear that you’ll use whatever resources you have to financially support her through the fallout for the actions she takes, but likely next step is to advise her to delete the program from everywhere. If “deleting” means physically destroying hardware, do it. If there’s time and there’s value in it, advise her to make it look like an “accident”. If she needs to loop in 1-2 extra people at the company to make this happen, do it, and extend them the same offer of financial support. Remove all barriers that might stop someone from preventing the release of unaligned AGI.
However also make it clear that there are limits. Don’t kill the CEO if you get desperate, for example. In general, destruction of property is fine in defense of humanity and history will likely look favorably on her one day. Killing or harming people is not.
I started showing symptoms and testing positive for COVID on Saturday. I’m now over nearly all the symptoms other than some pain in parts of my body and fatigue.
The curious question in my mind is, what’s causing this pain and fatigue and what can be done about it?
My high-level, I’m-not-a-doctor theory is that there’s something like generalized inflammation happening in my body, doing things makes it worse, and then my body sends out the signal to rest in order to get the inflammation back down. Once it’s down I can do things for a while until it builds up again and I have to stop. This is based on roughly the idea that it just feels like things are a bit sore but then it gets better if I rest and then comes back from doing nothing much more other than moving, sitting, standing, etc.
So maybe just taking anti-inflammatory drugs would help? I’ve done that when the pain got bad enough, but I didn’t notice it doing anything like letting me go longer without needing to rest.
I’m not yet in anything like long-COVID territory here since it’s not even been a week, so perhaps it’s reasonable to say I just need to keep resting and let my body heal, which I think is good advice to be sure, but I also want to put out a feeler to see if there’s something more I could be doing that would be helpful.
Sorry for leaving a comment after only reading the summary, so maybe this is addressed in the text, but I think I have a more concrete version of what I read as the theory being falling into a trap of a local maximum.
CFAR is just too weird.
I know lots of people like weird, but weird is self-limiting. And I don’t mean cute, “lol i have so many plants, i’m so weird”, within the normal person overton window weird, but proper normal-people-won’t-really-understand-you weird.
One of the great lessons I’ve learned from my years of Zen practice is “don’t talk about the weird”. There that means stuff like don’t talk about enlightenment, what happens during meditation (except with your teacher or a close dharma friend), or the things you can only know by experiencing them for yourself. It’s a distraction and only rarely helpful. Mostly you need to just keep at the everyday practice of Zen.
I claim rationality needs the same lesson. Lots of this stuff about rationality is actually, properly weird. And for the kind of person who enjoys it, they want to lean into the weird. This is a mistake. This kind of weird is for adepts and teachers to talk shop about on rare occasion. Everyday folks need to hear about the normal, everyday practice of a thing in order to do it and have it relate to their life.
Weird seems fine to Less Wrong because it’s tiered: you can find the weird on all posts, the normal on curated (or at least relatively normal). CFAR, to really succeed at what I see as its mission (bring rationality to the masses), needed to be the most normal version of rationality possible and as best I can tell it totally failed at this (e.g. CFAR seems single handedly responsible for dozens of bits of new jargon that could have been talked about in normal language without jargon).
The good news is I think it’s possible to start over if someone wanted. There’s some evidence this could work. For example, in the last 20 years cognitive bias training has become common. A lot of it is BS presentations rather than actual training, but there’s roots of rationality stuff out in the water among normal folks working jobs in big corporations. So we have some proof-of-concept that this is possible, but it requires optimizing for understanding by normal people, not the sort of weird people who are willing to spend their own money to get better at rationality.
(Note: I think CFAR succeeded in some important other ways. Those are out of scope for this comment, though.)
As it happens, I think this is a rather important topic. Failure to consider and mitigate the risk of assumptions creates both false negative (less concerning) and false positive (most concerning) risks when attempting to build aligned AI.
Lots of people are pretty suspicious of doing things that are too much like direct positive psychology work. They have their own traumas and hang ups about these sort of things and are pretty opposed to anything that looks even vaguely like group therapy. You can only do that kind of stuff if you’re the leader and you’re willing to have people quit over it (and, similarly, going forward are willing to use it as a differentiator when hiring).
What helps is just doing everyday things to create psychological safety. Listen to people. Don’t tease and taunt, even though this normally serves as a bonding activity. You can do it, but only very judiciously with people you know already feel secure and even then don’t do it in front of the people who aren’t secure. If you see other people doing things that will make others feel less psychologically safe, push back to make the workplace safer. Psychological safety is the foundation needed for development to happen.
Create space for people to try things and fail. Encourage people to take on tasks outside their comfort zone. Give them something like 70% work that they can succeed and at 30% work that they might fail at but will learn something trying. Let people have ownership and give them time to act on it. If they don’t after months, nudge them. Give them ideas. Encourage them to go out on a limb. Explicitly tell them they’re safe and you won’t hold it against them.
Part of the issue is like that celebrity, as wikipedia approaches the word, is broader than just modern TV, film, etc. celebrity and instead includes a wide variety of people who are not likely to be exceptionally attractive but are well known in some other way. There’s individual preferences in terms of who they think are attractive, but many politicians, authors, radio personalities, famous scientists, etc. are not conventionally attractive in the way movie stars are attractive and yet these people are still celebrities in a broad sense. However, I’ve not dug into the depths of wikipedia to see if, for example, this gap you see holds up if looking at pages that more directly talk about the qualities of film stars, for example.
AlphaGo is fairly constrained in what it’s designed to optimize for, but it still has the standard failure mode of “things we forgot to encode”. So for example AlphaGo could suffer the error of instrumental power grabbing in order to be able to get better at winning Go because we misspecified what we asked it to measure. This is a kind of failure introduced into the systems by humans failing to make adequately evaluate as we intended, since we cared about winning Go games while also minimizing side effects, but maybe when we constructed we forgot about minimizing side effects.
Really liking this model. It seems to actually deal with the problem of embeddedness for agents and the fact that there is no clear boundary to draw around what we call an agent other than one that’s convenient for some purpose.
I’ve obviously got thoughts on how this is operationalizing insights about “no-self” and dependent origination, but that doesn’t seem too important to get into, other than to say it gives me more reason to think this is likely to be useful.
Thanks for the app suggestions. I use Pocket today but as you noted it has some limitations and can be glitchy at times. I’ll give your suggested apps a try!