I’m a very confused person trying to become less confused. My history as a New Age mystic still colors everything I think even though I’m striving for rationality nowadays. Here’s my backstory if you’re interested.
MSRayne
Karma: 779
That actually would also be worthwhile. We will have AGI soon enough, after all, and I think it’s hard to argue that it wouldn’t be sentient and thus deserving of rights.
I appreciate the argument for clarifying principles, but I’m still not quite sure exactly how you think it’s best to find out what they are, or to write a principle-declaration statement. What all goes into such a thing? Are there lists of principles to pick and choose from? Or a pattern language for building your own?
Ah, I see. By the way, I didn’t intend to sound hostile or condescending but I may have done so, in which case I’m sorry. I think it’s evidence you have potential as a writer, and it’s probably longer than anything I’ve ever written lol (I’m terrible at finishing things) - I just would have preferred you go through a revision phase or two before publishing it, and considered how to make the characters a bit more realistic. (For instance, it seems like he adjusts psychologically to the transformation much faster than a normal person would, and suddenly becomes a hero when previously he was just a kid.)
Another possible moral, btw, is that the freedom to develop oneself and experiment with alternative modes of being and organization is part of what makes us human—not our mere body plan. There is a case to be made that the people of Cloud Nine are more human than the people of the Founder’s perfect city.
This is… interesting. I’m not exactly sure what the point is though, and it feels like fanfiction that was written in a stream of consciousness with no revision or planning. In particular I find it hard to believe that an earth-swallowing sea of maximizer AI nano would not… you know, actively do anything other than wait for people to fall into it. 5⁄10.
The funny thing is that the widespread attempts to make everything child-safe have backfired and made the entire world child-hostile.
It looks like most of my thinking (which, due to perfectionism and laziness, I have not really published anywhere) is about number 3 - characterizing target behavior—and some of 1 as well. I think a lot about what kind of world I actually want to steer towards (3), and that has led me to try to understand the real nature of the things that are ethically relevant (1), like the “self”, rights, etc. But as you say I often do feel a bit guilty about not trying hard on 2 and 4. I don’t think I really have the mind for that; I’m more a designer than a dev.
That seems like a bad move. Banning people from being human is always counterproductive. Any successful media of the future will be filled with weird porn; nobody wants to be censored.
I think we maybe shouldn’t reify “concepts” at all. To some extent it seems to me like there are only patterns of internal and external behavior. Certain stimuli—experiences, feelings, trains of thought—lead to certain sequences of words being produced. The question should not be “what does the word mean?” but rather “what does this word having been used say about the internal state of the one using it, in this context?” Words don’t have fixed meanings—they are entirely contextual.
Concepts are not things that actually exist as stable units—our usage of a finite set of words makes us think that they do, but in reality every pattern of neural activations is unique and we reach for the words that seem the least far away in some vector space (generated by training on our past experience of the same words in other contexts) to the position of the thought we actually want to express. Every such action moves the positions of those words a little bit in other people’s semantic spaces, leading to semantic drift—something like a rise in entropy, a gas expanding through a room.
Some people, such as myself, are not good at working with others, and end up in the Terry Davis position by default. Only my “knowledge” is mostly abstract philosophical theorizing that I constantly worry is of no use to anyone, and that I have hitherto never been able even to organize into a book or wiki others can read—though I have tried and am trying. What would your advice be to people who 1. have no money with which to start making more money and 2. have limited social skills and get extremely anxious and overwhelmed whenever they try to coordinate with others on any kind of shared project?
The problem here seems to be that these “democratic” socialists still believe in the legitimacy of the state, which is a mistake. What would actually happen if they get what they want is the center of elite power moving from the capitalist-controlled market to the equally capitalist-controlled government; in practice nothing would change, as corporate lobbyists and networks of cronyism already determine who gets elected and all the regulatory agencies are quickly captured by the industries they regulate. I think these people are well-intentioned, but nowhere near radical enough.
Also, as is typical for liberals, they confuse the market for capitalism, when in reality a capitalist market is only one possible kind of market, and a quite degenerate and unfree one at that—characterized by rampant wage slavery, not because markets intrinsically result in inequality (they don’t), but due to the existence of a state with a monopoly on violence, which is used by the capitalist class to back up their claims to “own” the means of production (i.e. to steal the products of workers’ labor out of their hands and give them a pittance in return); an anticapitalist market, freed from excessive rents, interest, and profits, along the lines of what mutualists envision creating, seems to me like it would be much more conducive to human flourishing.
I’m not an expert on it, but I’d encourage you to read about anarchist economic theories, particularly mutualism. Capitalism versus thinly-disguised state communism is a false dichotomy; the way I see it, the state, even if supposedly democratic in nature (as if the ability of a majority to coerce a minority is a sign of moral advancement rather than yet another example of the brutality of the natural state! and we all know most Western countries are only apparently democratic anyway—in reality controlled by corrupt elites who manipulate the public’s perception of reality to maintain their position), is not only unnecessary, but actively harmful to human freedom, and the ideals of the Enlightenment can only be attained when it, and capitalism, are both abolished in favor of local direct democracy (involving communes and worker cooperatives) and freed markets.
Also, check out the history of anarcho-communism. Everywhere it’s been tried, it has succeeded, or at least not decayed into totalitarianism—though it’s unfortunately always managed to get tried only in dangerous areas and times in the world like the Spanish civil war and (if democratic confederalism is considered an offshoot) the autonomous cities of northeast Syria today. State communism a la Soviet Russia and China is the one that should be left behind in the 20th century, along with states generally. Anarchism, in all its forms, deserves a closer look.
Many young people in the US are now attracted to socialism, an ideology that should have been left behind in the 20th century.
What makes you say this? The word “socialism” is so overloaded, I can’t tell which specific interpretation of it you dislike and why. I support some ideologies that describe themselves as socialist, but repudiate others.
Weird! How did I not notice that?
Well, I still think my pessimism was warranted given the epistemic state I had when I made the comment. I still don’t have a clue what the game is about though, and I think that’s a legitimate question… or did I miss an explanation of that on the website too? :/
I don’t think I remember seeing any of this information on the front page of the website. If it’s not there, it maybe ought to be?
This reminds me of optimization at a distance.
This gives me an idea… I wonder if there might be some way to use prediction markets to make safety, maintenance, monitoring etc short-term profitable for corporations (and maybe even governments), thereby incentivizing them to do it more and not cut costs?
The reason stuff like that gets cut when nothing is going on, is that it seems to be, in the short term, nothing but a drain of money, even though in the long term it’s saving money—so is there some way to make those long term gains available now so that they can be optimized for?
Does any of you actually have game design or development experience, or at least e.g. years of daydreaming and failed prototypes behind you? I’d like to point out that making a video game, particularly an MMO (which it looks like you’re thinking of), is very hard, and there are many problems that can ruin their success quite apart from the actual quality of the game.
You also haven’t said anything about the actual design itself yet; my prior, from lots and lots of personal experience, is to expect people who are making a Great Game That Will Help Change The World!!! to have tons of Cool Ideas and no discipline and to try to turn it into a kitchen sink and lose all momentum. (I’ve been those people. Over and over again.)
So: what are the actual game mechanics? What do players actually do in your game? And what makes that action fun?
looks at your website
Oh dear. You don’t even have a job listing for a game designer. And lots of talk about the AI you want to make, and as far as I can see, nothing about the game you want to make. Your goals are admirable; but my prediction as of right now is ~80% that you will be shocked at the difficulty of the undertaking and your own lack of readiness for it and give up within a year.
My suggestion is that you help other game studios that already exist build AI for their games in return for their agreement to allow you to use those contexts to explore the alignment-relevant questions, rather than attempting to make games yourself. I think that would be much, much more likely to succeed and require far less effort and struggle on your part.
Evidence that my opinion ought to matter to you: I’ve wanted to make games since I read Chris Crawford’s “On Game Design” when I was 11, and I have had a PERFECT EPIC GAME in my head for years that constantly changes form and never settles down, and I’ve never managed to actually build even a small prototype without giving up in frustration and self hate. I’ve also been part of several “small, passionate” teams of people who’d never made a game before and were trying to do so, two of which actually had a professional software developer at the head, and all of them failed miserably within a month or two with nothing to show for it.
It’s important to note that the only person who can determine how much a situation satisfies their values, is that person. I think a lot of alignment proposals have as a subtle underlying assumption that someone (or some set of someones) is going to be making decisions for everyone else. I think a world where that is the case has already been lost. I’m not sure to what extent these suggestions are examples of that, but they feel similar to me.
Btw, your vision of utopia on your website sounds roughly identical to mine which I’ve been thinking about for roughly ten years now. I am bad at writing in an organized way (and I constantly improve my understanding of things) so I’ve never written it up, but it might be nice for you and I to talk. In particular I’ve lately been thinking deeply about the ethics of consent and the need for counterfactual people (particularly the unborn and the dead) to have enforced rights—and the structures you call “nonperson forces” (I’d call them egregores) I consider to possibly have moral rights as well. I would really love to be able to think about all this stuff with you.
This shows a blind spot in my thinking (or in other people’s?) - I’ve always passively assumed we’d end up turning the whole universe into computronium and living entirely as uploads anyway, so I never would have thought “hey, let’s separate the program from the implementation”—I only ever have thought about the program (what virtual world I actually want to get instantiated) and never about the implementation to begin with—nor did I ever consider that people might have values over the state of the physical world itself, rather than over their own subjective experience. (Well, it’s obvious bioconservatives would, but I don’t expect them to win the universe.)
Are there any shapes of mind you think don’t have much to offer alignment, or will have unusual hindrances making it more difficult?
Lovely! I’m glad to hear it’s making sense to you. I had a leg up in perceiving this—I spent several years of my youth as a paranoid, possibly schizotypal occultist who literally believed in spirits—so it wasn’t hard for me, once I became more rational, to notice that I’d not been entirely wrong. But most people have no basis from which to start when perceiving these things!