Mostly the first reason. The “made of atoms that can be used for something else” piece of the standard AI x-risk argument also applies to suffering conscious beings, so an AI would be unlikely to keep them around if the standard AI x-risk argument ends up being true.
Nate Showell
Karma: 218
It’s worth noting that no reference to preferences has yet been made. That’s interesting because it suggests that there are both 0P-preferences and 1P-preferences. That intuitively makes sense, since I do care about both the actual state of the world, and what kind of experiences I’m having.
Believing in 0P-preferences seems to be a map-territory confusion, an instance of the Tyranny of the Intentional Object. The robot can’t observe the grid in a way that isn’t mediated by its sensors. There’s no way for 0P-statements to enter into the robot’s decision loop, and accordingly act as something the robot can have preferences over, except by routing through 1P-statements. Instead of directly having a 0P-preference for “a square of the grid is red,” the robot would have to have a 1P-preference for “I believe that a square of the grid is red.”
What’s your model of inflation in an AI takeoff scenario? I don’t know enough about macroeconomics to have a good model of what AI takeoff would do to inflation, but it seems like it would do something.
You’re underestimating how hard it is to fire people from government jobs, especially when those jobs are unionized. And even if there are strong economic incentives to replace teachers with AI, that still doesn’t address the ease of circumvention. There’s no surer way to make teenagers interested in a topic than to tell them that learning about it is forbidden.
All official teaching materials would be generated by a similar process. At about the same time, the teaching profession as we know it today ceases to exist. “Teachers” become merely administrators of the teaching system. No original documents from before AI are permitted for children to access in school.
This sequence of steps looks implausible to me. Teachers would have a vested interest in preventing it, since their jobs would be on the line. A requirement for all teaching materials to be AI-generated would also be trivially easy to circumvent, either by teachers or by the students themselves. Any administrator who tried to do these things would simply have their orders ignored, and the Streisand Effect would lead to a surge of interest in pre-AI documents among both teachers and students.
Why do you ordinarily not allow discussion of Buddhism on your posts?
Also, if anyone reading this does a naturalist study on a concept from Buddhist philosophy, I’d like to hear how it goes.
An edgy writing style is an epistemic red flag. A writing style designed to provoke a strong, usually negative, emotional response from the reader can be used to disguise the thinness of the substance behind the author’s arguments. Instead of carefully considering and evaluating the author’s arguments, the reader gets distracted by the disruption to their emotional state and reacts to the text in a way that more closely resembles a trauma response, with all the negative effects on their reasoning capabilities that such a response entails. Some examples of authors who do this: Friedrich Nietzsche, Grant Morrison, and The Last Psychiatrist.
OK, so maybe this is a cool new way to look at at certain aspects of GPT ontology… but why this primordial ontological role for the penis?
“Penis” probably has more synonyms than any other term in GPT-J’s training data.
I particularly wish people would taboo the word “optimize” more often. Referring to a process as “optimization” papers over questions like:
What feedback loop produces the increase or decrease in some quantity that is described as “optimization?” What steps does the loop have?
In what contexts does the feedback loop occur?
How might the effects of the feedback loop change between iterations? Does it always have the same effect on the quantity?
What secondary effects does the feedback loop have?
There’s a lot hiding behind the term “optimization,” and I think a large part of why early AI alignment research made so little progress was because people didn’t fully appreciate how leaky of an abstraction it is.
The “pure” case of complete causal separation, as with civilizations in separate regions of a multiverse, is an edge case of acausal trade that doesn’t reflect what the vast majority of real-world examples look like. You don’t need to speculate about galactic-scale civilizations to see what acausal trade looks like in practice: ordinary trade can already be modeled as acausal trade, as can coordination between ancestors and descendants. Economic and moral reasoning already have elements of superrationality to the extent that they rely on concepts such as incentives or universalizability, which introduce superrationality by conditioning one’s own behavior on other people’s predicted behavior. This ordinary acausal trade doesn’t require formal proofs or exact simulations—heuristic approximations of other people’s behavior are enough to give rise to it.
There are some styles of meditation that are explicitly described as “just sitting” or “doing nothing.”
Trust and distrust are social emotions. To feel either of them toward nature is to anthropomorphize it. In that sense, “deep atheism” is closer to theism than “shallow atheism,” in some cases no more than a valence-swap away.
An actually-deeply-atheistic form of atheism would involve stripping away anthropomorphization instead of trust. It would start with the observation that nature is alien and inhuman and would extend that observation to more places, acting as a kind of inverse of animism. This form of atheism would remove attributions of properties such as thought, desire, and free will from more types of entities: governments, corporations, ideas, and AI. At its maximum extent, it would even be applied to the processes that make up our own minds, with the recognition that such processes don’t come with any inherent essence of humanness attached. To really deepen atheism, make it illusionist.
Is trade ever fully causal? Ordinary trade can be modeled as acausal trade with the “no communication” condition relaxed. Even in a scenario as seemingly causal as using a vending machine, trade only occurs if the buyer believes that the vending machine will actually dispense its goods and not just take the buyer’s money. Similarly, the vending machine owner’s decision to set up the machine was informed by predictions about whether or not people would buy from it. The only kind of trade that seems like it might be fully causal is a self-executing contract that’s tied to an external trigger, and for which both parties have seen the source code and verified that the other party have enough resources to make the agreed-upon trade. Would a contract like that still have some acausal element anyway?
I agree: the capabilities of AI romantic partners probably aren’t the bottleneck to their wider adoption, considering the success of relatively primitive chatbots like Replika at attracting users. People sometimes become romantically attached to non-AI anime/video game characters despite not being able to interact with them at all! There doesn’t appear to be much correlation between the interactive capabilities of fictional-character romantic partners and their appeal to users/followers.
Sculpture wouldn’t be immune if robots get good enough, but live dance and theater still would be. I don’t expect humanoid robots to ever become completely indistinguishable from biological humans.
I agree, since dance and theater are already so frequently experienced in video form.
The future you’re describing only applies in Looking-At-Screens World. In sculpture, dance, and live theater, to name a few, human artists would still dominate. If generative AI achieved indistinguishability from human digital artists, I expect that those artists would shift toward more concrete media. Those concrete media would also become higher-status due to still requiring human artists.
I was comparing it to base-rate forecasting. Twitter leads people to over-update on evidence that isn’t actually very strong, making their predictions worse by moving their probabilities too far from the base rates.
I’ve come to believe (~65%) that Twitter is anti-informative: that it makes its users’ predictive calibration worse on average. On Manifold, I frequently adopt a strategy of betting against Twitter hype (e.g., on the LK-99 market), and this strategy has been profitable for me.
It seems like fixed points could be used to replace the concept of utility, or at least to ground it as an inferred property of more fundamental features of the agent-environment system. The concept of utility is motivated by the observation that agents have preference orderings over different states. Those preference orderings are statements about the relative stability of different states, in terms of the direction in which an agent tends to transition between them. It seems duplicative to have both utilities and fixed points as two separate descriptions of state transition processes in the agent-environment system; utilities look like they could be defined in terms of fixed points.
As one preliminary idea for how to do this, you could construct a fully connected graph in which the vertices are the probability distributions that satisfy . The edges are beliefs that represent hypothetical transitions between the fixed points. The graph would take the place of a preference ordering by describing the tendency of the agent to move between the fixed points if given the option. (You could also model incomplete preferences by not making the graph fully connected.) Performing power iteration with the transition matrix of would act as a counterpart to moving through the preference ordering.
Further exploration of this unification of utilities and fixed points could involve connecting to the beliefs that are actually, rather than just counterfactually, present in the agent-environment system, to describe what parts of the system the agent can control. Having a way to represent that connection could let us rewrite the instrumental constraint to not rely on .
Spoilers for Fullmetal Alchemist: Brotherhood:
Father is a good example of a character whose central flaw is his lack of green. Father was originally created as a fragment of Truth, but he never tries to understand the implications of that origin. Instead, he only ever sees God as something to be conquered, the holder of a power he can usurp. While the Elric brothers gain some understanding of “all is one, one is all” during their survival training, Father never does—he never stops seeing himself as a fragile cloud of gas inside a flask, obsessively needing to erect a dichotomy between controller and controlled. Not once in the series does he express anything resembling awe. When Father finally does encounter God beyond the Doorway of Truth, he doesn’t recognize what he’s seeing. The Elric brothers have artistic expressions of wonderment toward God inscribed on their Doorways of Truth, but Father’s Doorway of Truth is blank.
Father’s lack of green also extends to how he sees humans. It never seems to occur to Father that the taboo against human transmutation is anything more than an arbitrary rule. To him, humans are only ever tools or inconveniences, not people to appreciate for their own sake or look to for guidance. Joy-in-the-Other is what Father most deeply desires, but he doesn’t recognize this need.