All these people who vibecode agents with Claude Code now and addicted to it...
avturchin
Karma: 5,028
What I actually observed here is the popularity of books and movies about mass murders—but exactly I don’t like them.
In my introspection experience, I have both things and they are different. When I was in kindergarden (and was abused there) I wanted nuclear war to destroy it—and make me free. In school I wrote a novel about how all human disappeared from earth. But I also have a lot of more normal phantasies about gun battles, tanks etc.
I do think that people want to be mass murderers – but this desire is suppressed very well. There are only a few movies in which nobody is killed. There are other activities – like hunting, doomscrolling, or video games – that channel this suppressed desire so well that we have only a few real serial killers.
My view is that the desire to kill, as well as the desire to destroy the world, is built into most people, but it is also effectively suppressed and openly expressed only by some individuals or in special situations. The main problem is that a suppressed desire can change our behavior without our noticing. In the field of x-risks, this means that some decision-makers may make riskier bets than would be rational.
The problem which can be solved with money – along with many others problems. The meta-problem is that we will run out of money soon if we will try solve all problems with money. (Or attention, time or any other currency.)
The problem which is extremely important along with a group of other extremely important problems, and we can’t choose which one to start—or have to constantly switch between them and losing most energy on switching. (eg x-risks? personal health? pay next rent? etc). Often the number of very important problem is larger than attention span which results in complete halt of activity.
The problem about which I completely do not know what to do and generating ideas doesn’t help but eats energy.
The problem which I don’t have moral courage and energy to approach (eg go and make a test for some dangerous disease).
The problem of this view is that put emotions in upper level of human cognitive abilities. Stoics would not agree. Why not reason or spirit? Human emotions are rather mechanical and unconscious process inherited from our animal ancestors.
Also to reminds how my ex used to say: “This poem is bad!”
Me: “But maybe you wanted to say that you don’t like it?”
“No! It is objectively bad! And if you don’t see it, you are blind.”
There are more. One is outsider Widener art works devoted to future AI. The problem is that this evolved not into systematic efforts for AGI alignment but into dark art of putting hidden prompts somewhere in internet like Pliny the Liberator did.
I wrote a Message to any Future AI which has a goal to aligned a superintelligent AI.
James Miller et al wrote: An Appeal to AI Superintelligence: Reasons to Preserve Humanity.
Roco’s RB was also attempt to control superintelligent AI.
So there is a line of efforts in that way but I am not sure anyone continue to work on it. This approach works only for superintelligent AI, and includes the ideas like acausal deals and philosophical landmines.
alignment
Because both essentially are the same thing: alignment is about AI doing what I want—and capabilities are about Ai doing what I want?
I think there will be some reduction, but bit will not change the outcome significantly.
1 First, there will be opposite process—old brain will affect young body by different hormones and different behavioral habbits (like no running). Also, old brain may have some dormant cancer cells from previous body which will propagate through young body.2 Beneficial impact of young body will not affect all types of brain aging. May be 50 per cent slower aging which gives only one doubling or 7 years, but they will be mostly compensated by damage described in points 1 and 3.
3 The transfer procedure, if not required nanotechnology, will include a period of some brain ishemia or cooling, which will damage brain to some extent. And if we have advance nanotech, we don’t need new biological body at all—we can grow new one through old one.
BTW, we are starting soon an analog of LessWrong about life extension, may be you will be interested to share your post there: https://antimortality.org/.
To match our coherent experiences, the successor BB ’s need to have coherent memories, which only a few would.
Yes, it is similar to MWI—there infinitely many possible futures for me-now and most are crazy and discoherent. But we put Born rule above this distribution and select most coherent ones.
I think that there is some selection for stabilization mechanism in such observer-moment chains. Mueller suggested that it is similarity - or low Kolmogorov complexity—which creates penalty for less similr to now observer-moments.
Gompertz law is really brutal. Let’s assume that brain diseases causes 25 per cent of all death. At age 70 doubling of mortality is 7 years, so brain transfer reduces two doublings and gives +14 yeas to life expectancy.
However, around 105 years old doubling time is 1 year, so brain transfer will give only 2 years of life expectancy. Even if we reduce brain disease share in aging-related deaths to 1 per cent, the brain-transfer will give only around +7-8 years for centenaries, so 110 years is absolute limit for such technology. We have to address Gompertz law instead.
Doesn’t work because aging affects the brain as well and many deadly diseases originate in brain: insults, Alzheimer and brain cancers. Add risks of the transfer itself and the uncertainty when it should be done: too early for healthy aged person and and too late for one with cancer of any type (risks of metastases). All together it gives less than 10 years increase of of medium life expectancy.
In your words, there is a energy, concept of energy, and there is arbitrary representation of the concept of energy by letter E. The same way for colors: there is a real wavelength, the functional reaction to color in the brain and its arbitrary representation by redness inside consciousness.
After that follows rather technical moment which needs to be consider before we can discuss the question of existence of possible qualia. This technical moment is connected with the idea that unobserved qualia still have definite qualitative state. This seems to be so obvious that is often ignored, but its is an assumption which need to be considered.
Chalmers suggested that dancing qualia thought experiment—that a person can have different representations of colors in different moments of time but will not be able to observe that as functional part will remain the same. I mentioned here that such experiment assumes non-observed- qualia have definite qualitative state.
Blue-green abundance is functional part of qualia.
Exactly opposite is true for qualia—it will extraordinary claim to say that we know something about qualitative aspect of bird’s qualia.
Sorry for typos, I am not native speaker and didn’t use AI for editing here.
Maybe we should expect first AI-enabled takeover of rogue countries.
My point about apples was an illustration that’s some properties remains in possible things, this includes both shape and numbers, but not the first step of generalisation.
We are far from enumerating the qualitative aspect of qualia and may be it is even not possible in a provable way. Maybe because enumerating assumes that we can predict qualitative property of qualia which contradict idea that qualia depends only on itself.
Red has functional part and qualitative part, reddnes. I tried to explain this in my previous post about qualia as internal variables. For example E is energy in E=mc2. Its functional role is defined by the equation, but the variable itself can be denoted by a different letter, like W. Letters can also exist not related to the equations, that is free-floating letters.
Only if we assume that unobserved qualia do not reddness, when speaking about possible qualia’s color become meaningless, as by our assumption unobserved qualia don’t have redness and possible qualia is unobserved.
When I say that possible 25 has the same properties as real 25 I mean 25 apples as real 25 - and 25 as just a number as possible 25. Both 25 and 25 apples can be represented as 5x5. This property doesn’t depend on modal status of 25.
The distinction between open and closed qualia gives us an interesting hint why dancing qualia thought experiment is wrong (though how experiment can be wrong? only its conclusions). Chalmers seems to ignore this and implicitly assumes that there is no difference between open and closed qualia. As soon as we add this distinction, the experiment boils down to the question—are closed qualia the same as open ones.
It is rather objective fact that we can’t compare human’s and bird’s color qualia as birds have 4 primary colors. Birds colors are permanently closed to us. It is not just language game like in God’s definition.
Here we arrive to the interesting question—which properties remain in possible apples and in qualia?
For example, property of edibility will not remain in a possible apple, but a property being sphere-like remains in a possible apple.More generally, the properties related to the outside world disappear, but internal properties remain. If qualia are property related to the existence, they disappear in possible objects.
I see a possible source of alignment error when we specify preferences manually. If we take a real person—we get what such person really want, but it is also kinky, unstable etc. If we generate a set of nice sounding human values they may be good but unrelated to real humans and thus misaligned.
There are several midfiles available. There is a group of people who experimenting with creating open-sourced mindfiles of themselves, first of all, me and RomanS (and recently created mind model of Igor Kiriluk). There also several people who are very well documented: Leo Tolstoy, Anna Frank. The problem here is that they lived before AI and have a set of different world-views, especially Tolstoy.A good thing is that Tolstoy writings and dairies are deep into training set of most LLMs and can be called with short prompt.
This is what me and group of other people doing In the project called sideloading. We have a group in Telegram. We developed a tech to create a surprisingly good mind models, both static and with memory. We also think it will be helpful in AI safety.