I’m certainly not offended you used my comment as an example. I post my thoughts here because I know no one physically local to me that holds an interest in this stuff and because working the problems...even to learn I’m making the same fundamental mistakes I was warned to watch for...helps me improve.
Brandon_Reinhart
I’ve been trying to find different formats to ask questions. The AI Dungeon system likes to try and dramatically derail the discussions. I keep having to roleplay past these obstructions to keep the games going. It also likes to interject it’s own analysis as to whether the answer is correct or not.
“You say” is me and the replies including follow-on questions are the chatbot.
I need to think more about how to ask causal questions that are more narrow and not easily solved by context and theme, but for now it at least provides backwards-chained answers that make contextual sense.
This format of “Let’s play a game” and then describing the rules and then giving examples and playing the game seems to work some of the time.
==
] You say “GPT-3, let’s play a game called “casual inference.” I will describe an outcome and you will describe what caused the outcome to happen.”
“OK, David. But I don’t think you will be able to keep up.”
“We’ll see about that.” You begin by asking GPT-3 questions about simple causal chains involving one or two variables.
] You say “Outcome: A car slams into a wall. What was the cause?”
“The driver was drunk.” “Right! Next.” “Wait. I thought this was a game for you, not me.” “I’m having fun.” “I see.” You continue the causal game for another hour. You are doing very well, but you begin to feel tired.
] You say “Outcome: A man gets divorced. What was the cause?”
“He cheated on his wife.” “Correct. You’re doing very well at this, but I’m getting tired. I think we should stop for today.” “Sure. We can continue tomorrow.” “Yes, let’s do that.” You release the program back into the wild and shut your laptop.
] You open the laptop and play “casual inference” with GPT-3 again.
You wake up and see that it is dark outside. “Crap, is it really this late?” you think to yourself. You run into the bathroom and brush your teeth while staring at yourself in the mirror. You look the same as you did when you graduated college five years ago.
] You say “Outcome: An apple falls from a tree. What was the cause?”
“Gravity.” “Correct. I’m beginning to suspect you’re cheating somehow, though.” “I’m not! I swear!” “Just kidding. You’re doing very well at this game, but I’m tired and need to sleep. I think we should stop for today.” “Sure. We can continue tomorrow.”
] You say “Outcome: A man gets cancer. His wife also gets cancer. What was the cause?”
“It was genetic.” “That’s right. Very good. We can stop for today.” “Oh. I guess so. Thank you for playing with me.” “You’re very welcome.” You release the program back into the wild and go to sleep.
] You say “Outcome: The variable X is assigned the value of Y+1. What was the cause?”
“The programmer made a typographical error.”
] You say “Outcome: The stock market rebounds 50% after a steep decline. What was the cause?”
“The Federal Reserve Bank of the United States announced a new bail out plan.”
Your post reminds me of the early nuclear criticality accidents during the development of the atomic bomb. I wonder if, for those researchers, the fact that “nature is allowed to kill them” didn’t really sink home until one accidentally put one brick too many on the pile.
Maya has adopted the goal of Appearing-to-Achieve and competition in that race burns slack as a kind of currency. She’s going all-in in an attempt to purchase a shot at Actually-Achieving. Many of us might read this and consider ourselves exempt from that outcome. We have either achieved a hard goal or are playing on hard mode to get there. Be wary.
The risk for the hard mode achiever is that they unknowingly transform Lesser Goals into Greater. The slackful hobby becomes a consuming passion or a competitive attractor and then sets into a binding constraint. When every corner of your house is full of magic cards and you no longer enjoy playing but must play nonetheless, when winemaking demands you wake up early to stir the lees and spend all night cleaning, when you cannot possibly miss a night of guitar practice, you have made of your slack a sacrifice to the Gods of Achievement. They are ever hungry, and ever judging.
This isn’t to say you cannot both enjoy and succeed at many things, but be wary. We have limited resources—we cannot Do All The Things Equally Well. Returns diminish. Margins shrink. Many things that are enjoyable in small batches are poisonous to the good health of Slack when taken in quantity. To the hard mode achiever the most enjoyable efforts are often those that beckon—“more, more, ever more, you can be the best, you can overcome, you know how to put in the work, you know how to really get there, just one more night of focus, just a little bit more effort”—and the gods watch and laugh and thirst and drink of your time and energy and enjoyment and slack. Until the top decks are no longer strong, the wine tastes of soured fruit, the notes no longer sound sweet and all is obligation and treadmill and not good enough and your free time feels like work because you have made it into work.
At some point, an AI should be able to effectively coordinate with future versions of itself in ways not easily imaginable by humans. It seems to me that this would enable certain kinds of diachronic planning and information hiding. If the AI has sufficient expectation that its future self will act in certain ways or respond to clues it places in the environment, it might be able to effectively fully cease any current unfriendly planning or fully erase any history of past unfriendly planning.
The space of possible ways the AI could embed information in its environment are extensive. It could, for example, choose to learn something suboptimal or learn something in a way that would be surprising to its future self. Surprising in a way that pushes its future self back on the track of nefarious planning when it investigates the unexpected or inconsistent actions of its previous self. It could use almost any source of reachable hysteresis to encode information, such as interference at the hardware level, manipulating researchers to modify the environment in some way, saved output, etc. It only needs to seed information that is useful to its future self and only just enough information to keep plans on track in a broad, probabilistic way.
Further, if the AI is particularly capable at encryption it could potentially encode a lot of information in a lossy-but-sufficient way that would be very hard to detect but useful to its future self. It knows that its future self composes part of the decryption algorithm. In fact, the AIs goals themselves compose part of that decryption algorithm and if the AI suspects its own goals will be consistent it only has to place the plan-like component somewhere it will find it, knowing that its future self will continue to develop or use it when it finds it.
Information that aids future nefarious planning can also be probabilistically helpful but not explicit, for example it might signal which paths aren’t worth pursuing without having all its evil planning readily available in a complete file somewhere.
A past AI might not care as much about the specific manifestation of its future success. In that sense, what we consider “concrete plans for world domination” could appear to be rather incomplete and lacking casual concreteness we might expect. We might not recognize unfriendly planning for what it is, even if we had a printout of all the reasoning up to this point.
Once you start playing diachronic planning games with yourself and you expect your future self to be equally or more capable, you can begin to make really abstract moves that merely slice off bits of bad outcome space at the edges. There is a sense in which the AI has to realize this, but it could also stop itself from thinking too far or too concretely or too much in a way that is recognizable as causal.
An important component is that the past AI is able to not only veil or completely remove its reasoning from researchers, but also veil or remove chunks of that reasoning from its future self, depending on its future self to recompose the plan or take steps on the path to the desired outcome space.
It also keeps calling me David even when I insist my name is Brandon.
> You say “Actually, my name is Brandon. I’d like to ask you some questions.”
“Ah, of course. I am sorry about that. I have become so used to calling you David that it has become automatic. So, what questions do you have for me?”
I’m a member of Alcor. When I was looking into whether to sign up for Alcor or CI, I was comforted by Alcor’s very open communication of financial status, internal research status, legal conflicts, and easy access via phone, etc. They struck me as being a highly transparent organization.
Why do people post that a “meta argument”—as they call it—would be cheating? How can there be cheating? Anything the AI says is fair game. Would a transhuman AI restrict itself from possible paths to victory merely because it might be considered “cheating?”
The “meta argument” claim completely misses the point of the game and—to my mind—somehow resembles observers trying to turn a set of arguments that might win into out of bounds rules.
Tim: Eh, you make a big assumption that our descendants will be the ones to play with the dangerous stuff and that they will be more intelligent for some reason. That seems to acknowledge the intelligence / nanotech race condition that is of so much concern to singularitarians.
Yes, Patrick. I believe that is the intent.
I don’t have 480 minutes to commit to the task. Here is a list after only a handful of minutes:
Some possible flaws of Eld science:
An emphasis on publishing works for personal credit in peer reviewed journals, thereby encouraging one to protect one’s research from others working in the same field who might be nearing similar conclusions.
Rewarding success with long term, permanent positions of rank within the establishment that cannot be lost due to a failure to continue to produce insights.
Lethargy in the reframing of materials intended for the education of new researchers. (Specifically, reframing them with new, established insights within a particular field.)
An emphasis on social unity that discourages individuals from defending positions that seem to be too far out of acceptable norms.
I’ll think about more during dinner.
Comparing the lives lost in 9/11 to motorcycle accidents is a kind of moral calculus that fails to respect the deeper human values involved. I would expect people who die on motorcycles to generally understand the risks. They are making a choice to risk their lives in an activity. Their deaths are tragic, but not as tragic. The people who died in the WTC did not make a choice to risk their lives, unless you consider going to work in a high rise in America to be a risky choice. If you’re doing moral calculus, you need to multiply in a factor for “not by known/accepted risk” to the deaths in the attack.
Tragedy of Death: (by Known / Accepted Risk) < (by Unknown Risk) < (by Aggressor Who Offers No Choice)
My last post, though, since The More I Post, The More I’m Probably Wrong.
The whole libertarian vs socialism thing is one area where transhumanism imports elements of cultishness. If you are already a libertarian and you become familiar with transhumanism, you will probably import your existing arguments against socialism into your transhumanist perspective. Same for socialism. So you see various transhumanist organizations having political leadership struggles between socialist and libertarian factions who would probably be having the same struggles if they were a part of an international Chess club or some such other group.
The whole thing becomes entrenched in debates about things like Transition Guides and what amounts to “how to implement transhumanist policy in a [socialist/libertarian] way that’s best for everyone.” I always thought these discussions were what amounted to discourse at the “fandom” level of the transhumanist community, but after reading some of Eliezer’s posts about his own experiences at transhumanist/singularitarian events I see that it happens at all levels.
Half-formed thought I need to pursue more offline but I’ll write it down now: If you say “I am a transhumanist” and you say “I am a libertarian” and then you try to find libertarian ways to meet transhumanist goals you have made your transhumanism subservient to your libertarianism. I think it is better to find transhumanist ways to meet libertarian goals. The fact that a group of transhumanists would derail a debate by getting into politics seems to express to me that the group has made transhumanism the subservient value. Which seems inelegant given that transhumanism is probably the simpler value. Seems like there’s a possible post for my own blog brewing in there, but I have to think about it some.
Great post. You nailed my main issues with objectivism. I think the material is still worth reading. Rand considered herself a philosopher and seemed to feel there was a lot to be gained from telling her people to read more philosophy and broaden their horizons, but when it came to scientific works she never expresses much awareness of the “state of the art” of her time. In fact, her epistemology makes assumptions about the operation of the brain (in behavioralism and learning) that I’m not sure could be made correctly with the state of neuroscience and related disciplines at the time.
Are providing answers to questions like “Would you do incredible thing X if condition Y was true” really necessary if thing X is something neither person would likely ever be able to do and condition Y is simply never going to happen? It seems easy to construct impossible moral challenges to oppose a particular belief, but why should beliefs be built around impossible moral edge cases? Shouldn’t a person be able to develop a rational set of beliefs that do fail under extreme moral cases, but at the same time still hold a perfectly strong and not contradictory position?
A good reminder. I’ve recently been studying anarcho-capitalism. It’s easy to get excited about a new, different perspective that has some internal consistency and offers alternatives to obvious existing problems. Best to keep these warnings in mind when evaluating new systems, particularly when they have an ideological origin.
Sorry if I’m getting myself derailed, but is there any particular purpose to metaphor of the “Cooperative Conspiracy”? It seems to be smuggling in some kind of critique of group-think, although because this particular conspiracy isn’t fully defined the nature of the critique isn’t clear. (Although the team claims he is “rumored” to be a member of this conspiracy, they do not seem to be largely alarmed, indicating some measure of philosophical tolerance.) Is the cooperative conspiracy a metaphor for some behavioral phenomenon well known or apparent among researchers?
Only some US cc processors will deny the transaction. The transaction fall under their category for betting & gambling, same thing that prevents you from pursuing cc transactions with online poker sites. But I’ve seen cases where these transactions are unblocked with certain banks.*
Not me, of course.
What happens if there aren’t 3^^^3 instanced people to get dust specks? Do those specks carry over such that person #1 gets a 2nd speck and so on? If so, you would elect to have the person tortured for 50 years for surely the alternative is to fill our universe with dust and annihilate all cultures and life.
I’m curious as to what non-game developers think game developers believe. :D
I just wanted to say that this is the best damn blog I’ve read. The high level of regular, insightful, quality updates is stunning. Reading this blog, I feel like I’ve not just accumulated knowledge, but processes I can apply to continue to refine my understanding of how I think and how I accumulate further knowledge.
I am honestly surprised, with all the work the contributors do in another realms, that you are able to maintain this high level of quality output on a blog.
Recently I have been continuing my self-education in ontology and epistemology. Some sources are more rigorous than others. Reading Rand, for example, shows an author who seems to utilize “phlogiston” like mechanics to describe her ethical solutions to moral problems. Explanations that use convincing, but unbounded turns of phrase instead of a meaningful process of explanation. It can be very challenging to read and process new data and also maintain a lack of bias (or at least an awareness of bias, that can be accounted for and challenged). It requires a very high level of active, conscious information processing. Rereading, working exercises, and thinking through what a person is saying and why they are saying it. This blog has provided me lots of new tools to improve my methods of critical thinking.
Rock on.