On the Nature of Agency
Epistemic status: Fairly high confidence. Probably not complete, but I do think pieces presented are all part of the true picture.
Agency and being an agent are common terms within the Effective Altruist and Rationalist communities. I have only used heard them used positively, typically to praise someone’s virtue as an agent or to decry the lack of agents and agency.
As a concept, agency is related to planning. Since I’ve been writing about planning of late, I thought I’d attempt a breakdown of agency within my general planning paradigm. I apologize that this write-up is a little rushed.
Examples of Agency and Non-Agency
Keeping the exposition concrete, let’s start with some instances of things I expect to be described as more or less agentic.
Things likely to be described as more agentic:
Setting non-trivial, non-standard (ambitious) goals and achieving them.
Dropping out of school and founding a startup.
Becoming president.
Making a million dollars.
Self-teaching rather than enrolling in courses.
Building and customizing your own things rather than purchasing pre-made.
Researching niche areas which you think are promising instead of the popular ones.
Disregarding typical relationship structures and creating your own which work for you.
Noticing there are issues in your workplace and instigating change.
Noticing there are issues in your society and instigating change.
Gaming the system (especially when the system is unjust).
Reading diverse materials to form your own models and opinions.
Being able to receive a task with limited instruction and able to execute it competently to a high standard.
Ignoring conventional advice and inventing your own better way.
Having heretical thoughts and believing things other people think are wrong or crazy.
Strong willingness to trust one’s own opinion despite the views of others and even experts.
Accomplishing any difficult task which most people are unable to.
Things likely to be described as less agentic:
Spending your life working on the family farm like your parents before you.
Unquestioningly adopting the views of your friends, family, faith, or other authorities.
Proceeding through school, undergraduate, graduate degree as the simplest pathway.
Sticking to conservatively prestigious or stable professions like medicine, law, nursing, construction, teaching, or even programming.
Seeking social approval for one’s plans and actions or at least ensuring that one’s actions do not leave the range of typically socially-approved actions.
Prioritizing safety, security, and stability over gambits for greater gain.
Requiring specific direction, instruction, training, or guidance to complete novel tasks.
Following current fads or what’s in fashion. Generally high imitation of others.
I have not worked hard to craft these lists so I doubt they are properly comprehensive or representative, but they should suffice to get us on the same page.
At times it has been popular, and admittedly controversial, to speak of how some people are PCs (player characters) and others are mere NPCs (non-player characters). PCs (agents) do interesting things and save the day. NPCs (non-agents) followed scripted, boring behaviors like stock and man the village store for the duration of the game. PCs are the heroes, NPCs are not. (It is usually the case that anyone is accomplished or impressive is granted the title of agent.)
The Ingredients of Agency
What causes people in one list to be agentic and those in other to be not so? A ready answer is that people being agentic are willing to be weird. The examples divide nicely along conformity vs nonconformity, doing what everyone else does vs forging your own path.
This is emphatically true—agency requires willingness to be different—but I argue that it is incidental. If you think agency is about being weird, you have missed the point. Though it is not overly apparent from the examples, the core of agency is about accomplishing goals strategically. Foremost, an agent has a goal and is trying to select their actions so as to accomplish that goal.
But in a way, so does everyone. We need a little more detail than this standard definition that you’ve probably heard already. Even if we say that a computer NPC is mindlessly executing their programming, a human shopkeeper legitimately does have their own goals and values towards which their actions contribute. It should be uncontroversial to say that all humans are choosing their actions in a way that digital video game NPCs are not. So what makes the difference between a boring human shopkeeper and Barack Obama?
It is not that one chooses their actions and the other does not at all, but rather the process by which they do so.
First, we must note that planning is really, super-duper, fricking hard. Planning well requires the ability to predict reality well and do some seriously involved computation. Given this, one of the easiest ways to plan is to model your plan off someone else’s. It’s even better if you can model your plan of those executed by dozens, hundreds, or thousands of others. When you choose actions already taken by others, you have access to some really good data about will happen when you take those actions. If I want to go to grad school, there’s a large supply of people I could talk to for advice. By imitating the plans of others, I ensure that I probably won’t get any worse results than they did, plus it’s easier to know which plans are low-variance when lots of people have tried them.
The difference is that agents are usually executing new computation and taking risks with plans that have much higher uncertainty and higher risk associated. The non-agent gets to rely on the fact that many people’s model thought particular actions were a good idea, whereas the agent much more needs to rely on their own models.
Consider the archetypical founders dropping out of college to work on their idea (back before this was a cool, admirable archetype). Most people were following a pathway with a predictably good outcome. Wozniak, Jobs, and Gates probably would have graduated and gotten fine jobs just like people in their reference class. But they instead calculated that a better option for them was to drop out with the attendant risk. This was a course of action that stemmed from them thinking for themselves what would most lead towards their goals and values. Bringing their own models and computation to the situation.
This bumps into another feature of agency: agents who are running their own action-selection computation for themselves rather than imitating others (including their past selves) are able to be a lot more responsive to their individual situation. Plans made my the collective have limited ability to include parameters which customize the plan to the individual.
Returning to the question of willingness to be weird: it is more a prerequisite for agency than the core definition. An agent who is trying to accomplish a goal as strategically as possible and who is running new computation and performing a search for the optimal plan for them—they simply don’t want to be restricted to any existing solutions. If an existing solution is the best, no problem, it’s just that you don’t want to throw out an optimal solution just because it’s unusual.
What other people do is useful data, but to an agent it won’t inherently be a limitation. (Admittedly, you do have to account for how other people will react to your deviance in your plans. More on this soon.)
Mini-summary: an agent tries to accomplish a goal by running relatively more of their own new computation/planning relative to pure imitation of cached plans of others or their past selves; they will not discard plans simply because they are unusual.
Now why be agentic? When you imitate the plans of others, you protect against downside risk and likely won’t get worse than most. On the other hand, you probably won’t get better results either. You cap your expected outcomes within a comfortable range.
I suspect that among the traits which cause people to exhibit the behaviors we consider agentic are:
A sense that more is possible. They believe that there are reachable outcomes much better than the existing default.
An aspiration, striving, or ambition to the more which can they envision.
Conversely, complacency is the enemy of agency.
There has to be something which makes a person want to invest the effort to come up with their own plans rather than marching along the beaten paths with everyone else.
Or maybe not, maybe some people have powerful and active minds so that it’s relatively cheap to them to be thinking fresh for themselves. Maybe in their case, the impetus is boredom.
An agent must believe that more is possible, and more crucially they must believe that it possible for them to cause that more. This corresponds to the locus of control and self-efficacy variables in the core self-evaluations framework.
Further, any agent whose significant work you’re able to see has likely possessed a good measure of conscientiousness. I’m not sure if lazy geniuses might count as an exception. Still, I expect a strong correlation here. Most people who are conscientious are not agents, but those agents you who observe are probably conscientious.
The last few traits could be considered “positive traits” active traits that agents must possess. There are also “negative traits”, traits that most people have and agents must have less of.
Agents strive for more, but the price they pay is a willingness to risk getting even less. If you drop out of college, you make millions of dollars or you might end up broke and without a degree. When you make your own plans, possibly go off the beaten path, there is likelihood of failure. What’s worse, if you fail then you can be blamed for your failure. Pity may be withheld because you could have played it safe and gone along with everyone else, and instead, you decided to be weird.
Across all the different situations, agents might be risking money, home, respect, limb, life, love, career, freedom and all else they value. Not everyone has the constitution for that.
Now, just a bit more needs to be said around agents and the social situation. Above it was implied that the plans are of others are essentially orthogonal to those of an agent. They’re not limited by them. That is true as far as the planning process goes, but as far as enacting one’s plans goes, it takes a little more.
An agent doesn’t just risk that their unusual plans might fail in ways more standard plans don’t, they also have to risk they will 1) lose out on approval because they are not doing the standard things, 2) actively be punished for being a deviant with their plans.
If there is status attached to going along certain popular pathways, e.g. working in the right prestigious organizations, then anyone who decides to follow a different plan that only makes sense to them must necessarily forego status they might have otherwise attained. (Perhaps they are gambling that they’ll make more eventually on their own path, but at least at first they are foregoing.) This creates a strong filter that agents are those people who were either indifferent to status or willing to sacrifice it for greater gain.
Ideally it would only be potentially foregone status which would affect agents, instead there is the further element that deviance is often actively punished. It’s the stereotype that the establishment strikes out against the anti-establishment. Everyone group will have its known truths and its taboos. Arrogance and hubris are sins. We are hypocrites who simultaneously praise those who have gone above and beyond while sneering at those who attempt to do the same. Agents must have thick skin.
Indeed, agents must have thick skin and be willing to gamble. In contrast, imitation (which approximates non-agency) serves the multifold function of a) saving computation, b) reducing risk, and c) guarding against social opprobrium and even optimizing for social reward.
Everyday Agents
I fear the above discussion of agency has tended too grandiose, too much towards revolutionaries and founders of billion dollar companies. Really though, we need agency on much more mundane scales too.
Consider that an agentic employee is a supremely useful employee since:
If you give them a task with limited instruction, they will use their own new computation/planning to figure out how to execute it well. They don’t need things step by step.
They will supply their own sense that more is possible and push for excellence.
They will take initiative to make things better because of their sense that more is possible.
They will not be inhibited by excessive fear of failure or your disapproval because they did something other than follow explicit instructions.
They’re willing to take on new and unusual tasks and learn new skills because:
They have high self-efficacy
They’re in the habit of thinking their own fresh thoughts instead of imitating and enacting ready-made plans.
They’re willing to fail in the course of trial-and-error to figure things out.
An agentic employee is the kind of employee who doesn’t succumb to defensive decision-making.
Why Agency is Uncommon
The discussion so far can be summarized neatly by saying what is which makes agency uncommon:
Agency is rare because it involves planning for yourself, going off the beaten the path rather than imitating and copying the plans of others or your past self. Planning for yourself is really, really hard.
It requires the skill of planning for yourself.
It requires the expenditure of effort to do so.
Agency requires both a sense that more is possible and a striving to reach that more. Conversely, agency is poisoned by the presence of complacency.
Agency requires belief in one’s self-efficacy and that one is the locus of control in their life.
Agency requires lower than average risk-aversion since attempting potentially non-standard plans means risking non-standard failure.
In particular, it requires low social risk-aversion.
This applies at both the macro and micro scale.
Agency requires conscientiousness.
Agency requires a resilience to social sacrifice either passively via foregone status or approval or actively via the punishment received for deviating from the norm.
Agent/Non-Agent More and Less Agentic
This post is primarily written in terms of agents and non-agents. While convenient, this language is dangerous. I fear that when being an agent is cool, everyone will think themselves is one and go to sleep each night congratulating themselves for being an agent unlike all those bad dumb non-agents.
Better to treat agency is a spectrum upon which you can be scoring higher or lower on any given day.
How agentic was I today?
Was I being too risk-averse today?
Was I worrying too much about social approval?
Am I trying to think with fresh eyes and from first principles of new ways I could accomplish my goals? Or am I just rehashing the same possibilities again and again?
Addendum: Mysterious Old Wizards
A friend of mine has the hypothesis that a primary way to cause people to be more agentic is to have someone be their mysterious old wizard a la Gandalf, Dumbledore, and Quirrell. A mysterious old wizard shows up, believes in someone, and probably says some mysterious stuff, and this help induces agency.
I can see this working. This might have happened to me a bit, too. If someone shows up and is sufficiently high-status in your mind, and they tell you that you are capable of great things, they can cause all the following:
You allow yourself to believe more is possible because the wizard believes it too.
You believe that you are capable (self-efficacy, locus of control) because the wizard does.
You are willing to go on your quest despite social opprobrium because now you only care about the society of you and the wizard, not anyone else.
I can see it working.
- What does it take to defend the world against out-of-control AGIs? by (25 Oct 2022 14:47 UTC; 194 points)
- How do people become ambitious? by (4 Apr 2019 19:12 UTC; 59 points)
- What does it take to defend the world against out-of-control AGIs? by (EA Forum; 25 Oct 2022 14:47 UTC; 43 points)
- Agency and Sphexishness: A Second Glance by (16 Apr 2019 1:25 UTC; 25 points)
I’m wondering if you are using these terms as synonyms for conformism/non-conformism, or if there is more to being agentic than refusing to conform and looking for your own way?
Also this SSC post seems relevant. Scott calls them “thinkers”.
There is much more to being agentic than nonconformity. I apologize for unusual rambliness of this post. I can highlight where I tried to express this:
I would add that it seems like you’ve focused your entire thought process on the problem of how “rationality” works. And you’ve also discussed the problem of how you should get rationality done.
I am not sure to what extent you think that I can think of any reasonably useful and desirable parts of rationality which your proposal doesn’t actually consider.
+1 good summary. I mean, you can always set a five minute timer if you want to think of more reasonably useful and desirable parts of rationality.
For this to work, you need to have enough time (usually after you have a reasonable amount of experience) with other rationality techniques to learn what you have.
In order for that you must have some amount of background knowledge of how to actually implement your claims, that includes explicit communication, face-to-face communication, explicit communication about the thing that “seems” to match. If you have that amount of knowledge, you can easily have a problem with people being too shy to actually have the kind of conversations they like (e.g. PUA jargon).
And you must be somewhat lacking in social skills.
If you happen to be a little shy, then you’ll have a problem with people being overly shy.
I have the impression that people who can find a lot of social skills out of a group can often become very social and are unable to overcome the obstacles they face. (I could be too shy, but I’d really like a “how can you show that you won’t be shy without being afraid”?)
In short, people can easily be oblivious to social challenges for longer than they need to overcome them. For example, the first hit with a mouse at a bar is a challenge to overcome. The other person will give a lot of lectures in their bar and some social skills, although the most useful ones are the ones that create the social challenge for the other person.
While I acknowledge this, which I see as good advice, I don’t see why it should apply to everyone, or even the most powerful people. If, for instance, some people have social skills that are fairly rare, so that they’re not able to overcome their social skills, then that is a factor of a two.
I guess if you wanted to be successful as a social worker in a social setting, that could be more. If you wanted to be successful as a social worker in a social setting then you probably used more social skills than you needed, and that seems to be your excuse.
I mean, yeah, agents (like everyone) benefit from social skills.
You say that people should not be allowed to have their preferences but that they should just have their beliefs (e.g., to have preferences is to have a larger utility)
But there are some important questions which I think are not answered here:
Does this apply to a human brain [1], or to any other species [2] that we aren’t part of? [...]
[...]
In what sense does our decisions make sense if we don’t have a conscious mind?
Too real, GPT2, too real.
The problem of having a conscious mind seems like the single most useful thing. The whole “be a conscious being” aspect seems very useful compared to the huge gap between conscious and unconscious minds which otherwise seem to be something like how the brain doesn’t have a conscious mind, but is pretty pretty far off from it.
Of course, you could also try other approaches. Your mind could be a computer or a CPU, or you could try some different approach.
I suggest that maybe having one mental brain that does the opposite something is more useful than one mental brain that “does both things”.
This seems relevant to Qiaochu’s interests.
I can’t think of anything else that is more relevant to Qiaochu and Qiaochu.
I think these are just examples. (A) the whole post is about the first time in the past that we lived in a civilization, and B) it’s probably easier “thinking” (with respect to people and societies) to “play with the environment, to become the kind of man who can survive the end”. (B) the whole post actually sounds like it’s going to have an advantage when it starts from “The Art of Super Strategy” and the “What is the art of human rationality” part.
I think it’s easier to understand the art of human rationality from its own standpoint. If people have the feeling that rationality is a field that we should have all practicing that is really impressive to us—and yet they think “hooray!” we give them the feel that rationality is a field that we should have all practicing and looking impressive to us—why, of course, should anyone want to practice it or are they too dumb to see? (A) as a general statement: “what does rationality teach us to do?” (B) as a type of self indication that a little bit of creativity could be useful.
To illustrate a bit of my own thought process, I might encourage you to check out Eliezer’s posts on the rationalist community and Alicorn’s posts on agency and all that. So, what I have found so far is that more people are willing to participate than would in my previous post on agency or Alicorn’s post (because of your cached self defense). Note that I don’t necessarily disagree with A or Alicorn’s thinking about agency and would like to avoid that.
Also note that Alicorn’s post has the truth sub- danger tag, so if I saw it there, I would also share it with the other commenters. But you have to be aware of the tag’s text, and even if you don’t, I certainly hope I haven’t made things worse. And I hope you don’t mind posting it.
Consider the possibility that you’re (and many are) conflating multiple distinct things under the term “agency”.
1) Moral weight. I’ll admit that I used the term “NPC” in my youth, and I regret it now. In fact, everyone has a rich life and their own struggles.
2) Something like “self-actualization”, perhaps “growth mindset” or other names for a feeling of empowerment and the belief that one has significant influence over one’s future. This is the locus-of-control belief (for the future).
3) Actual exercised influence over one’s future. This is the locus-of-control truth (in the past).
4) Useful non-comformity—others’ perceptions of unpredictability in desirable dimensions. Simply being weird isn’t enough—being successfully weird is necessary.
I’m not sure I agree that “planning” is the key element. I think belief (on the agent’s part and in those evaluating agency of others) in locus of control is more important. Planning may make control more effective, but isn’t truly necessary to have the control.
I’m not at all sure that these are the same thing. But I do wonder if they’re related in the sense that they classify into a cluster in an annoying but strong evolutionary strategy: “ally worth acquiring”. Someone powerful enough (or likely to become so) to have an influence on my goals, and at the same time unpredictable enough that I need to spend effort on cultivating the alliance rather than taking it for granted.
Conflating (or even having a strong correlation between) 1 the others is tricky because considering any significant portion of humanity to be “non-agents” is horrific, but putting effort into coordinating with non-agents is stupid. I suspect the right middle ground is to realize that there’s a wide band of potential agency, and humans occupy a narrow part of it. What seems like large variance to us is really pretty trivial.
“Makes sense, and humans don’t have any other simple agents. We have them out in the wild, we have them out in the wild, we don’t have them out in the wild . .
This comes from a post that makes reference to a real life case that doesn’t use the word “emotion.”
I liked the post. Some random thoughts I had while reading some of your random thoughts
You can have agency along the dimensions of both plans and goals. An employee can be very agenty within an organization but not think about whether climbing the organization makes sense. Someone can spend months trying to make sure the path they choose is one that will make them happy, and then simply follow the prescribed script on that path. This screams for a 2 by 2 grid.
On goals—You can be agenty with very big “save/conquer the world” goals, but it’s also possible to be agenty with much smaller goals around personal happiness/hedonism. In Venkatesh Rao’s Loser/Clueless/Sociopath framework, the sociopaths at the top are very agenty—they’ve used their own understanding to climb the ladder and shape the politics such that they win. However, the losers at the bottom are also very agenty—they’ve taken a look at the prizes, and decided the game is not worth playing, so instead just grab their paycheck and look through personal fulfillment in their hobbies and extracurricular activities. This seems to be a clear plan/goal agentiic split.
On status—How you view status depends very much on your goals. If you have large goals, status seems to be pretty instrumental to achieving most large goals, and so pursuing it makes sense. Even for small goals around personal fulfillment, status seems useful. However, the difference seems to be that more agenty people can take status as object, rather than subject. That is, the less agentic way to view status is to internalize shame, embarrassment, contempt, and other status based feelings as ends in themselves, whereas a more agenty person would view the way other people react to status as a means to an end.
I find the second one useful. At the same time, I think the third one is less useful and less likely to be useful as a discussion framework (it’d have a nice name, and the name gets less weird and useful if it’s backed up with a bunch of highly-voted posts from past LW articles). It might be useful to try to define ‘agenty’ for this post.
The problem with the first version of this post was that it was a poor, implicit post; the second version is more likely to get a lot of downvotes, because the post is clearly aimed at newcomers and not as a discussion forum.
I’m confused. Are you saying that highly-upvoted posts make a name nicer and therefore less useful? If so, can you describe the mechanisms behind this?
I think the most common response to ‘community’ should have been a post to LessWrong and its founding sequences. We wanted to create a place for rationalists that can discuss the Art and Science, at least this year.
A place to discuss an important topic which might otherwise not be discussed, is CFAR.
To paraphrase two of the core themes on this site:
In humans, the world is an incredible fabric of complex, entangled, self-reinforcing processes. (Many of us can be made aware of this fact, though usually it isn’t necessary.)
Rather than trying to collect information from each person, we use a series of simpler, more useful shared models, based on our conversations and experiences.
One of the CFAR concepts is the “agent-agent distinction”, where the AI agent is the AI agent, and so also tries to understand its own goals and limitations. One of the main motivations for the new Center for Applied Rationality is to build a sense of understanding and understanding of its own motivations, and these are attempts to make the AI general intelligent agents reflect humanity’s goals.
CFAR has a whole overarching mission of raising the sanity waterline. That is, it is attempting to create people who can benefit from thinking clearly, and help each other reach its goals while also being more effective. As a nonprofit, CFAR is close to being a place where we can help people overcome their irrational biases, and to do so as best they can.
CFAR is building a whole new rationality curriculum that will hopefully help people become more effective.
We are reviving this November and November again. Like the rest of the January 2008 Singularity Summits, we tweaking the curriculum and organization of CFAR alumni. The new thinking tools workshop will give us specific ways to apply the principles of rationality to the behavior of different groups or individuals, as opposed to mere human “capital” and organizational stuff. In past years, we’ve moved from “organizational inadequacy” to “additional common denominator” posts and to “additional organizational capital” posts, where I’d like there to be funding for doing high-impact good. Emphasizing and organizing such an organization allows us to step outside of the academic and organizational space that would normally be reserved for highly technical people.
In a more practical sense, the oxen-back infrastructure in Berkeley is existing, but we’
Ah. It’s a bot. I suppose the name should have tipped me off. At least I get Being More Confused By Fiction Than Reality points.
You’ve covered a lot of things in my writing and I enjoy this. Thanks for what you’ve done.
How did you write that in less than a minute?
Thanks for writing this!
The paper doesn’t show up until 4:30, even if the book is intended very specifically to convince a significant fraction of the population that cryonics is plausible for humanity.
For those that don’t understand, see here.
For the first chapter, you basically make the case that the scientific method is wrong, or at least that is not a strawman. The rest of what I’ve read is the most mainstream-seeming and obvious the scientific method seems to be no doubt wrong.
For the second chapter, you basically show the science in a book that is primarily about the ability of human minds to generalize from one another, where it is based on:
The basic Bayes-related questions of personal identity—i.e., how much should it be enough to have a psychological effect?
How much should one’s society be prioritised that one can be in this position?
In particular, it doesn’t fit in the Bostrom model of personal identity.
It’s not entirely clear that the subject matter of writing about the relationship between personal identity and mental identity is exactly the sort of information-theoretic question that could lead us to a useful answer, and the kind of information that would be better in the context of the question you will find yourself in the future.
You probably see this phrasing and the objections about science, and I think you’ve taken them too far. Yes, it’s hard to argue about the degree of overlap with the scientific method, and yes, the two are relevant. But if it’s going to work in such extreme cases for a long time, then there should be an additional thing called “substrategic knowledge”.
One of the things that I think is really important is to figure out how to think about personal identity under the “internal locus of control”. Here’s my attempt to begin that.
The “internal locus of control” seems like it would be quite a different subject in this context, I think from where I’ve heading and here.
If this doesn’t work, then there could be some fundamental difference between myself and a rationalist.
A few of my observations:
I’ve been a slow reader for a while now. I was probably under-remembering a lot about LW when I was a teenager, so I didn’t really get anything.
I was
I read this as an argument why black should win the conflict between black and green in the magic color relations. I am more familiar where black is painted as the viallain side and this contrast seemd to be very fruitful in my mind as this seemed like the rare point of view that is pro-black.
-Black does have to worry about punishments of deviants but black also can be quite okay about being in actual conflict. The “error corrections” of deviation punishments can sometimes be justified. At the local level you don’t always have the means to appriciate your actions consequences for the wider picture. Green likes empricially found global solutions and it really really dislikes when black individuals have a strong influence in their locality preventing certain types of solutions. Ignoring the athmospheric effects of CO2 allows for easier design of powerful industrial processes and picking up that fruit might be very relevant to personal interests but its not like the restriction (or I guess concern at this stage) is there without cause.
-Black takes actions that have known downsides that it can think it can get away with. The trouble is that sometimes there are downsides they could not have known from their start position and they can get bitten by things they could have known but did not in fact know. Green doesn’t have a model why what it does works so it handles unknown dangers equally as well as familiar threats. Curiosity like kills cats (althoguht curiosity isn’t selected against atleast not strong enough).
-In the magic metaphor the willingness to take loss is much more severe. It’s about willingness to cut your hand off to get at your goal. Framing it as “having high consitution” easily paints a picture where losses can be recovered from. But if you die or you lose your arm you don’t get resurrected or regrow a limb. Black is about achieving things that are otherwise impossible but also summoing stuff that would never happen to you otherwise too.
-The flip side of preventing taking others opinins too readily is imposing your will too strongly on others. If you take on a vocabulary that suggests and helps you make a plan of action but also demonises other people it can be easier to be the villain (pretty common trope also that villains act and heroes react). If it is better to rule hell and than serve in heaven is it worth the trouble to turn heaven into hell based solely that your personal situation improves? The whole “aligment problem” is kind of the realisation that an independent mind will have an independent direction which could theorethically be in conflict with other directions. The black stance is that “indidivual will” is a resource to be celebrated and not a problem to be solved away.
I have to be careful what I say about the model that I have in mind in that post. I just want to be clear that I don’t think we need this model in order to make a certain kind of assumption.
It is a universal law that all things in life are a kind of universal law.
A universal law is not some kind of universal law.
There are many ideas that seem to hold the idea that “anything in life” (say, human-caused global warming) is universal in this universe (for example, no heat gradient for humans), but many things in evolution can’t be universal in this universe (for example, no carbon dating for a human), even if we knew it’s universal law.
The model we presented in your post can, in some cases, be more fundamental than the one we actually actually have. But the “common” model, the one that I proposed in your post, just doesn’t hold any stronger claim.
I don’t think it’s a good model to model general conditions of development from which a universal law is in conflict with universal nature.