AI Alignment Researcher, working with organizations such as CEA, https://groundless.ai/, https://www.epistemic-experiments.org/ and others. You can find out about me at https://www.adityaarpitha.com/
Working on multiple projects reach out to me if you want to collaborate
My preference for direct messages or communication is Telegram or Signal
I tweet sometimes,
Make sure you RSVP here at the luma page—https://luma.com/sc7gyanb
Excited to see you all :)
Abram has done such an amazing job in making the agenda accessible, I really enjoyed reading this post. I keep procastinating but I hope to have my own version of “what I understand to be sahil’s agenda” out soon.
Until then I would love to jump on a video call with anyone who is confused but still wants to engage, we can talk about the agenda, maybe read through one of the posts in the live theory sequence. I have spend a lot of time around Sahil and think I have a decent grasp of his ideas.
We are raising funding for the Residency program happening in India! So if anyone wants to support us please check that project out.
You don’t save the world by doing inward-looking stuff at a monastery
My model of MAPLE is that this is exactly the crux. The theory of impact that they have is based on developing a potent enough ideology (a “meme”) that then spreads among powerful actors in the real world affecting their actions.
Currently they are in the design phase and are not looking to scale but eventually this “design a trustworthy religion” will become “offer the religion to cyborgs around the world”. Cyborgs are those humans who closely integrate AI, tech into their workflows, building a sort of exocortex.
If we don’t live in the FOOM world, then imagine how gradual disempowerment will play out. We will see increasing inequality in competence, productivity, capability as these models get more powerful. Also these cyborgs will be even more isolated, confused, lonely and due to suffering be keen to explore alternative religions to make sense of the world and decide on what motivations to pursue as meaningful, moral, etc.
It is these cyborgs that will first see consensus reality break down and traditional frameworks of meaning making will fail to help them properly orient to the world. Already we are seeing this with chatbots that are able to provide us “intimacy”, making us feel seen better than our friends. When algorithms can optimise our quantified self better than our own intuition, won’t more people abdicate their power to these systems?
The future of our species will depend on how empathetic, kind, compassionate, wise these cyborgs are. So I believe MAPLE is trying to design a lens to see the world, a perspective that resonates with them, that is true and ensures we continue to believe in truth being more than just predictive power, the ability to manipulate and control the external world, rather it is about living in harmony with other sentient beings.
They have an online course here which is likely more accurate than my interpretation. But during my stay with them as an AI fellow I got to interact with the students there and ask questions about why they believed the key was to cultivate the mind, how mind was chief.
Religions or ideologies are powerful because they form the framework by which we coordinate at scale. The current AI Alignment problem is not just a technical problem but a socio-technical problem, we are dealing with actors taking selfish actions, falling into race dynamics and feeling helpless due to the incentive structures around us.
In this video Soryu talks about how impactful religions have been in determining the flow of history. Towards the end he emphasises how people who deem themselves secular are in fact religious themselves. We have humanistic religions that hold human connection to be sacred and fundamental to meaning, scientific religions that hold reductionist, materialistic perspectives to be an axiomatic truth.
These systems make factual claims about what Is true about the world and then try to act innocent when normative claims (Ought) emerge (interdependently) as a consequence of believing in the factual claims. This relates to what herschel was saying about the culture being confused. If we believe that the basic ontology of reality is matter then concepts like love, joy, friendship will be reduced to transactional details, we can have allies that we cooperate with but we stop believing in anything valuable which cannot be reified. We seem to confused the map with the territory and believe that the representation can fully capture the referent.
I find all this very interesting and uncorrelated with other approaches to solve the alignment problem. I’d love to see more empirical falsifiable tests that could be designed to investigate the sort of moral realism adjacent (maybe?) implications of the claim that there is no hume’s gap.
I highly recommend people watch Connor talk about his interpretation of this post.
He talks about how Eliezer is a person who managed to access many anti memes that slid right off our heads.
What is an anti meme you might ask?
By their very nature they resist being known or integrated into your world model. You struggle to remember them. Just like how memes are sticky, and go viral anti memes are slippery and struggle to gain traction.
They could be extraordinarily boring. They could be facts about yourself that your ego protects you from really grasping. Facts about human nature that cause cognitive dissonance because it contradicts existing beliefs you hold dear.
Insights that are anti memes are hard to communicate. You need to use fables, narratives, embody the idea into a story and convey the vibe. You use metaphors in the story.
Jokes or humor is a great way to communicate anti memes, especially ones that are socially awkward and outside people’s overton window.
---
Then Connor gives the example of this post—Death with Dignity as an example of an anti meme. Most of those reading the post seem to completely miss the actual point even when it was clearly spelled out.
maybe it’s somewhat easier on account of how we have more introspective access to a working mind than we have to the low-level physical fields;
We have a biased access which makes things tricker because we weren’t selected for our introspection skills to be high fidelity and having a correspondence with reality. Rather it’s about the utility to survival.
It doesn’t have to be the result of explicit metaphysical beliefs...it could be the result of vague guesswork, and analogical thinking.
Yeah I could be wrong but my claim is implicit metaphysical beliefs have a big role here.
defining “agentic” as “possessing spooky metaphysical free will” rather than “not passive”. It’s perfectly possibly to build an agent-in-the-sense-of-active out of mechanical parts.
I was just noting that people who are aware of the internal workings of AI will have to acutely face cognitive dissonance if they admit it can have “spooky” agency. They can’t compartmentalize it the way others can.
“topics about which philosophy is still concerned because we don’t or can’t get information that would enable us to have sufficient certainty of answers to allow those topics to transition into science”.
I think that is quite close. I mean the implicit assumptions behind all these discussions, which are unquestioned. Moral realism, Computationalism, Empiricism, and Reductionism all come to mind. These topics cannot be tested or falsified with the scientific method.
but there’s not really anything here that seems like an argument that would convince anyone who didn’t already agree
I thought it would be best to try even if I am not confident it will make any impact on people reading it. My attempt is, like you rightly said, trying to get AI safety researchers to take philosophy more seriously. Most people see it as a past time that they can enjoy for intrinsic pleasure. In my opinion there is a lot of utility if we practiced going more meta until we could see the underpinnings of both the problem of x risk and the solution.
Some of the utility comes from being able to communicate it to more diverse people at higher fidelity. The rest comes from empowering existing researchers to maybe make a breakthrough in alignment itself.
A lot of these objects like values, and goals seem to exist strongly in our ontology. I would like to see people try and question these things, consider other possibilities.
This exchange between Connor and Joscha seems to be an example where Connor clearly is irritated at the question because it is trying to use philosophy to question if we should even both saving humanity, is humans bad by our own standards. I can understand how he feels completely. But notice how Joscha seems to seriously think the philosophy of what values we have and how they are justified are very important.
In this community it seems to taken as fact that the direction we align the AI towards is something to be considered after figuring out how to set the direction in anyway whatsoever. We have decoupled these two things. I would like to question these assumptions, and because I am not smart enough maybe others can also try. This needs us to unsee the boundaries we are so used to and be very careful which ones we put down.
In particular, they might unlearn it in narrow contexts related to their immediate work, but then get confused and fail to unlearn it in general, resulting in them getting confused about things like agency and free will.
Yeah, I was hoping to draw attention to this problem with my post. I love the embedded agency comic series. Yeah, the cartesian boundary is one of such boundaries which most of us have but again if we want to think about alignment honestly, I think it is worthwhile to train to unsee that too.
I will check out your book. I hope to also maybe write something that can help people grok monoism and other philosophical ideas they might want to consider in its entirety.
Aren’t non-academics and non-experts the majority,
I was talking about people who had not grokked materialism which is the majority. The people who are not aware of the technical details model AI as this black box, therefore, seem to be more open to considering that it might be agentic but that is them just deferring to an outside view that sounds convincing rather than building their own model.
so maybe people there, who are working on AI and machine learning, more often have a religious or spiritual concept of human nature, compared to their counterparts in the secularized West?
Most people I talked to were from India and it is possible there is a pattern there. But I see similar arguments come up even in the people in the west. When people say “it is just statistics”, they seem to be pointing to the idea that deterministic processes can never be agentic.
I am not trying to bring consciousness into the discussion necessarily but I think there is value in helping people make their existing philosophical beliefs more explicit so that they can see it to the natural conclusion.
Thanks for the constructive critism. I thought about it and I guess I need to increase the legibility of what I wrote.
I will add a TLDR and update the post soon.
Some things don’t make sense unless you really experience it. Personally I have no words for the warping effects such emotions have on you. It’s comparable to having kids or getting brain injury.
It’s a socially acceptable mental disorder.
The only thing is to notice when you are in that state and put very low credence on all positive opinions you have about your Limerent Object. You cannot know to a high confidence anything about them in that state. Give it a few years.
Don’t take decisions you can’t undo, entangle parts of your life which will be painful to detach later.
But it’s a ride worth going on. No point in living life too safely. Have fun but stay safe out there.
Evolution failed at imparting its goal into humans, since humans have their own goals that they shoot for instead when given a chance.
To me, your framing of inner misalignment sounds like Goodharting itself because we evolved our intrinsic motivations towards these measures because they were good measures in the ancestral environment. But when we got access to advanced technology we kept optimizing on the measure (sex, sugar, beauty, etc) which led to it becoming no longer a measure of the actual target (kids, calories, health, etc.)
I think outer alignment is better thought of as a property of the objective function i.e. “an objective function is outer aligned if it incentivizes or produces the behavior we actually want on the training distribution.”
You should come for the Bangalore meet-up this Sunday. If you are near this part of India.
I asked out my crushes. Worked out well for me.
I used to be really inhibited, now I have tried weed, alcohol and am really enjoying the moment.
Feels nice to see my name in a story. This fact about Romans is just so tasty.
It was hard to really imagine someone getting so emotionally caught up about a fact. I didn’t expect to find it so hard.
Most fights are never about the underlying fact but it’s tribal, about winning. If people cared about knowing the truth it would be discussions not debates.
This is totally possible and valid. I would love for this to be true. It’s just that we can plan for the worst case scenario.
I think it can help to believe that things will turn out ok, we are training the AI on human data. It might adopt some values. Once you believe that, then working on alignment can just be a matter of planning for the worst case scenario.
Just in case. Seem like that would be better for mental health.
Oh ok, I had heard this theory from a friend. Looks like I was misinformed. Rather than evolution causing cancer I think it is more accurate to say evolution doesn’t care if older individuals die off.
evolutionary investments in tumor suppression may have waned in older age.
https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3660034/
Moreover, some processes which are important for organismal fitness in youth may actually contribute to tissue decline and increased cancer in old age, a concept known as antagonistic pleiotropy
So thanks for clearing that up. I understand cancer better now.
When I talk to my friends, I start with the alignment problem. I found this analogy to human evolution really drives home the point that it’s a hard problem. We aren’t close to solving it.
https://youtu.be/bJLcIBixGj8
So at this time questions come up about how intelligence necessarily means morality. I talk about orthogonality thesis. Then why would the AI care about anything other that what it was explicitly told to do, the danger comes from Instrumental convergence.
Finally people tend to say, we can never do it, they talk about spirituality, uniqueness of human intelligence. So I need to talk about evolution hill climbing to animal intelligence, how narrow ai has small models while we just need AGI to have a generalised world model. Brains are just electrochemical complex systems. It’s not magic.
Talk about pathways, imagen, gpt3 and what it can do, talk about how scaling seems to be working.
https://www.gwern.net/Scaling-hypothesis#why-does-pretraining-work
So it makes sense we might have AGI in our lifetime and we have tons of money and brains working on building ai capability, fewer on safety.
Try practising on other smart friends and develop your skill, you need to ensure people don’t get bored so you can’t use too much time. Use nice analogies. Have answers to frequent questions ready.
I think this is how evolution selected for cancer. To ensure humans don’t live for too long competing for resources with their descendants.
Internal time bombs are important to code in. But it’s hard to integrate that into the AI in a way that the ai doesn’t just remove it the first chance it gets. Humans don’t like having to die you know. AGI would also not like the suicide bomb tied onto it.
The problem of coding this (as part of training) into an optimiser such that it adopts it as a mesa objective is an unsolved problem.
Same this post is what made me decide I can’t leave it to the experts. It is just a matter of spending the required time to catch up on what we know and tried. As Keltham said—Diversity is in itself an asset. If we can get enough humans to think about this problem we can get some breakthroughs many some angles others have not thought of yet.
For me, it was not demotivating. He is not a god, and it ain’t over until the fat lady sings. Things are serious and it just means we should all try our best. In fact, I am kinda happy to imagine we might see a utopia happen in my lifetime. Most humans don’t get a chance to literally save the world. It would be really sad if I died a few years before some AGI turned into a superintelligence.
I am so happy to read this post, knowing Jake is working on this makes me glad. For sure I will be nudging more people I know to apply. I hope Coefficient gets really high quality applicants in this round.
In fact whenever I am asking for help to pivot towards contributing to the field I use the OP’s technical AIS RFP as a resource to get people to think about what kind of experiments they would want to do or upskill to be able to do. so “But grantmakers don’t just evaluate proposals; we elicit them.” is really powerful and true.
This definitely makes the role highly impactful and the main update I had from this is that it is more fun/motivating than you would have imagined. Thanks for sharing.