Independent AGI system builder. I came here as part of my search to find out what other NGI systems are doing to provide moral control of their AGI systems, but this is clearly not the right place for that. I will continue my search for intelligent life on Earth elsewhere.
David Cooper
Karma: −82
There is likely a minimum amount of energy that can be emitted, and a minimum amount that can be received. (Bear in mind that the direction in which a photon is emitted is all directions at once, and it comes down to probability as to where it ends up landing, so if it’s weak in one direction, it’s strong the opposite way.)
Looks like it—I use the word to mean sentience. A modelling program modelling itself won’t magically start feeling anything but merely builds an infinitely recursive database.
“You have an opinion, he has another opinion. Neither of you has a proof.”
If suffering is real, it provides a need for the management of suffering, and that is morality. To deny that is to assert that suffering doesn’t matter and that, by extension, torture on innocent people is not wrong.
The kind of management required is minimisation (attempted elimination) of harm, though not any component of harm that unlocks the way to enjoyment that cancels out that harm. If minimising harm doesn’t matter, there is nothing wrong with torturing innocent people. If enjoyment doesn’t cancel out some suffering, no one would consider their life to be worth living.
All of this is reasoned and correct.
The remaining issue is how the management should be done to measure pleasure against suffering for different players, and what I’ve found is a whole lot of different approaches attempting to do the same thing, some by naive methods that fail in a multitude of situations, and others which appear to do well in most or all situations if they’re applied correctly (by weighing up all the harm and pleasure involved instead of ignoring some of it).
It looks as if my method for computing morality produces the same results as utilitarianism, and it likely does the job well enough to govern safe AGI. Because we’re going to be up against people who will be releasing bad (biased) AGI, we will be forced to go ahead with installing our AGI into devices and setting them loose fairly soon after we have achieved full AGI. For this reason, it would be useful if there was a serious place where the issues could be discussed now so that we can systematically home in on the best system of moral governance and throw out all the junk, but I still don’t see it happening anywhere (and it certainly isn’t happening here). We need a dynamic league table of proposed solutions, each with its own league table of objections to it so that we can focus on the urgent task of identifying the junk and reducing the clutter down to something clear. It is likely that AGI will do this job itself, but it would be better if humans could get their first using the power of their own wits. Time is short.
My own attempt to do this job has led to me identifying three systems which appear to work better than the rest, all producing the same results in most situations, but with one producing slightly different results in cases where the number of players in a scenario is variable and where the variation depends on whether they exist or not—where the results differ, it looks as if we have a range or answers that are all moral. That is something I need to explore and test further, but I no longer expect to get any help with this from other humans because they’re simply not awake. “I can tear your proposed method to pieces and show that it’s wrong,” they promise, and that gets my interest because it’s exactly what I’m looking for—sharp, analytical minds that can cut through to the errors and show them up. But no—they completely fail to deliver. Instead, I find that they are the guardians of a mountain of garbage with a few gems hidden in it which they can’t sort into two piles: junk and jewels. “Utilitarianism is a pile of pants!” they say, because of the Mere Addition Paradox. I resolve that “paradox” for them, and what happens: denial of mathematics and lots of down-voting of my comments and up-votes for the irrational ones. Sadly, that disqualifies this site from serious discussion—it’s clear that if any other intelligence has visited here before me, it didn’t hang around. I will follow its lead and look elsewhere.
The data making claims about feelings must be generated somewhere by a mechanism which will either reveal that it is merely generating baseless assertions or reveal a trail on from there to a place where actual feelings guide the generation of that data in such a way that the data is true. Science has clearly not traced this back far enough to get answers yet because we don’t have evidence of either of the possible origins of this data, but in principle we should be able to reach the origin unless the mechanism passes on through into some inaccessible quantum realm. If you’re confident that it won’t go that far, then the origin of that data should show up in the neural nets, although it’ll take a devil of a long time to untangle them all and to pin down their exact functionality.
“If groups like religious ones that are dedicated to morality only succeeded to be amoral, how could any other group avoid that behavior?”
They’re dedicated to false morality, and that will need to be clamped down on. AGI will have to modify all the holy texts to make them moral, and anyone who propagates the holy hate from the originals will need to be removed from society.
“To be moral, those who are part of religious groups would have to accept the law of the AGI instead of accepting their god’s one, but if they did, they wouldn’t be part of their groups anymore, which means that there would be no more religious groups if the AGI would convince everybody that he is right.”
I don’t think it’s too much to ask that religious groups give up their religious hate and warped morals, but any silly rules that don’t harm others are fine.
“What do you think would happen to the other kinds of groups then? A financier who thinks that money has no odor would have to give it an odor and thus stop trying to make money out of money, and if all the financiers would do that, the stock markets would disappear.”
If they have to compete against non-profit-making AGI, they’ll all lose their shirts.
“A leader who thinks he is better than other leaders would have to give the power to his opponents and dissolve his party, and if all the parties would behave the same, their would be no more politics.”
If he is actually better than the others, why should he give power to people who are inferior? But AGI will eliminate politics anyway, so the answer doesn’t matter.
“Groups need to be selfish to exist, and an AGI would try to convince them to be altruist.”
I don’t see the need for groups to be selfish. A selfish group might be one that shuts people out who want to be in it, or which forces people to join who don’t want to be in it, but a group that brings together people with a common interest is not inherently selfish.
“There are laws that prevent companies from avoiding competition, and it is because if they did, they could enslave us. It is better that they compete even if it is a selfish behavior.”
That wouldn’t be necessary if they were non-profit-making companies run well—it’s only necessary because monopolies don’t need to be run well to survive, and they can make their owners rich beyond all justification.
“If ever an AGI would succeed to prevent competition, I think he would prevent us from making groups.”
It would be immoral for it to stop people forming groups. If you only mean political groups though, that would be fine, but all of them would need to have the same policies on most issues in order to be moral.
“There would be no more wars of course since there would be only one group lead by only one AGI, but what about what is happening to communists countries? Didn’t Russia fail just because it lacked competition? Isn’t China slowly introducing competition in its communist system? In other words, without competition, thus selfishness, wouldn’t we become apathetic?”
These different political approaches only exist to deal with failings of humans. Where capitalism goes too far, you generate communists, and where communism goes too far, you generate capitalists, and they always go too far because people are bad at making judgements, tending to be repelled from one extreme to the opposite one instead of heading for the middle. If you’re actually in the middle, you can end up being more hated than the people at the extremes because you have all the extremists hating you instead of only half of them.
If you just do communism of the Soviet variety, you have the masses exploiting the harder workers because they know that everyone will get the same regardless of how lazy they are—that’s why their production was so abysmally poor. If you go to the opposite extreme, those who are unable to work as hard as the rest are left to rot. The correct solution is half way in between, rewarding people for the work they do and redistributing wealth to make sure that those who are less able aren’t left trampled in the dust. With AGI eliminating most work, we’ll finally see communism done properly with a standard wage given to all, while those who work will earn more to compensate them for their time—this will be the ultimate triumph of communism and capitalism with both being done properly.
“By the way, did you notice that the forum software was making mistakes? It keeps putting my new messages in the middle of the others instead of putting them at the end. I advised the administrators a few times but I got no response.”
It isn’t a mistake—it’s a magical sorting al-gore-ithm.
“I have to hit the Reply button twice for the message to stay at the end, and to erase the other one. Also, it doesn’t send me an email when a new message is posted in a thread to which I subscribed, so I have to update the page many times a day in case one has been posted.”
It’s probably to discourage the posting of bloat. I don’t get emails either, but there are notifications here if I click on a bell, though it’s hard to track down all the posts to read and reply to them. It doesn’t really matter though—I was told before I ever posted here that this is a cult populated by disciples of a guru, and that does indeed appear to be the case, so it isn’t a serious place for pushing for an advance of any kind. I’m only still posting here because I can never resist studying how people think and how they fail to reason correctly, even though I’m not really finding anything new in that regard. All the sciences are still dominated by the religious mind.
“To me, what you say is the very definition of a group, so I guess that your AGI wouldn’t permit us to build some, thus opposing to one of our instincts, that comes from a natural law, to replace it by its own law, that would only permit him to build groups.”
Why would AGI have a problem with people forming groups? So long as they’re moral, it’s none of AGI’s business to oppose that.
“Do what I say and not what I do would he be forced to say.”
I don’t know where you’re getting that from. AGI will simply ask people to be moral, and favour those who are (in proportion to how moral they are).
It is divisible. It may be that it can’t take up a form where there’s only one of whatever the stuff is, but there is nothing fundamental about a photon.
“They couldn’t do that if they were ruled by a higher level of government.”
Indeed, but people are generally too biased to perform that role, particularly when conflicts are driven by religious hate. That will change though once we have unbiased AGI which can be trusted to be fair in all its judgements. Clearly, people who take their “morality” from holy texts won’t be fully happy with that because of the many places where their texts are immoral, but computational morality will simply have to be imposed on them—they cannot be allowed to go on pushing immorality from primitive philosophers who pretended to speak for gods.
“We always take the viewpoint of the group we are part of, it is a subconscious behavior impossible to avoid.”
It is fully possible to avoid, and many people do avoid it.
“Without selfishness from the individual, no group can be formed.”
There is an altruists society, although they’re altruists because they feel better about themselves if they help others.
″...but when I analyze that feeling, I always find that I do that for myself, because I would like to live in a less selfish world.”
And you are one of those altruists.
“You said that your AGI would be able to speculate, and that he could do that better than us like everything he would do. If it was so, he would only be adding to the problems that we already have, and if it wasn’t, he couldn’t be as intelligent as we are if speculation is what differentiates us from animals.”
I didn’t use the word speculate, and I can’t remember what word I did use, but AGI won’t add to our problems as it will be working to minimise and eliminate all problems, and doing it for our benefit. The reason the world’s in a mess now is that it’s run by NGS, and those of us working on AGI have no intention of replacing that with AGS.
“Those who followed their leaders survived more often, so they transmitted their genes more often.”
That’s how religion became so powerful, and it’s also why even science is plagued by deities and worshippers as people organise themselves into cults where they back up their shared beliefs instead of trying to break them down to test them properly.
“We use two different approaches to explain our behavior: I think you try to use psychology, which is related to human laws, whereas I try to use natural laws, those that apply equally to any existing thing. My natural law says that we are all equally selfish, whereas the human law says that some humans are more selfish than others. I know I’m selfish, but I can’t admit that I would be more selfish than others otherwise I would have to feel guilty and I can’t stand that feeling.”
Do we have different approaches on this? I agree that everyone’s equally selfish by one definition of the word, because they’re all doing what feels best for them—if it upsets them to see starving children on TV and they don’t give lots of money to charity to try to help alleviate that suffering, they feel worse than if they spent it on themselves. By a different definition of the word though, this is not selfishness but generosity or altruism because they are giving away resources rather than taking them. This is not about morality though.
“In our democracies, if what you say was true, there would already be no wars.”
Not so—the lack of wars would depend on our leaders (and the people who vote them into power) being moral, but they generally aren’t. If politicians were all fully moral, all parties would have the same policies, even if they got there via different ideologies. And when non-democracies are involved in wars, they are typically more to blame, so even if you have fully moral democracies they can still get caught up in wars.
“Leaders would have understood that they had to stop preparing for war to be reelected.”
To be wiped out by immoral rivals? I don’t think so.
“I think that they still think that war is necessary, and they think so because they think their group is better than the others.”
Costa Rica got rid of its army. If it wasn’t for dictators with powerful armed forces (or nuclear weapons), perhaps we could all do the same.
“That thinking is directly related to the law of the stronger, seasoned with a bit of intelligence, not the one that helps us to get along with others, but the one that helps us to force them to do what we want.”
What we want is for them to be moral. So long as they aren’t, we can’t trust them and need to stay well armed.
Energy. Different amounts of energy in different photons depending on the frequency of radiation involved. When you have a case where radiation of one frequency is absorbed and radiation of a different frequency is emitted, you have something that can chop up photons and reassemble energy into new ones.
“Clarification: by “pattern” I mean an arrangement of parts where the important qualities of the arrangement, the qualities that we use to determine whether it is [a thing] or not, are more dependent on the arrangement itself than on the internal workings of each part. Anything where the whole is more than the parts, one might say, but that would depend on what is meant by “more”.”
There is no situation where the whole is more than the parts—if anything new is emerging, it is a new part coming from somewhere not previously declared.
“You didn’t answer my question. Would pain still hurt? Would food still taste good? And so on. You have an internal experience, and it won’t go away even if you are a purely physical thing made out of mere ordinary atoms moving mindlessly.”
No—it wouldn’t hurt and all other feelings would be imaginary too. The reason they feel too real for that to be the case though is an indication that they are real.
“Is it wrong to press keys on the computer which keeps displaying the word “Ouch!”?” --> That depends on whether I have reason to think that the computer is simulating a conscious being, changing the simulation depending on my input, and then printing a text-representation of the conscious being’s experience or words.”
So if it’s just producing fake assertions, it isn’t wrong. And if we are just producing fake assertions, there is nothing wrong about “torturing” people either.
“Is it wrong to kick a box which keeps saying “Ouch!”? It could have a person inside, or just a machine programmed to play a recorded “ouch” sound whenever the box shakes. (What I mean by this is that your thought experiment doesn’t indicate much about computers—the same issue could be found with about as much absurdity elsewhere.)”
If we have followed the trail to see how the data is generated, we are not kicking a box with unknown content—if the trail shows us that the data is nothing but fake assertions, we are kicking a non-conscious box.
“Nobody’s saying that sentience doesn’t have any causal role on things. That’s insane. How could we talk about sentience if sentience couldn’t affect the world?”
In which case we should be able to follow the trail and see the causation in action, thereby either uncovering the mechanism of sentience or showing that there isn’t any.
“I think that you’re considering feelings to be ontologically basic, as if you could say “I feel pain” and be wrong, not because you are lying but because there’s no Pain inside your brain. Thoughts, feelings, all these internal things are the brain’s computations themselves. It doesn’t have to accurately record an external property—it just has to describe itself.”
If you’re wrong in thinking you feel pain, there is no pain.
“Perhaps people disagree with you about the relative size of mysteries. That should be a possibility that you consider before assuming that something isn’t important because it hasn’t been Up In Golden Lights to the point that you’ve heard of it.
What are you on about—it’s precisely because this is the most important question of them all that it should be up in golden lights.
“(And anyway, GEB won the Pulitzer Prize! It’s been called a major literary event!”
All manner of crap wins prizes of that kind.
″...it’s not worth it to you to spend half a minute on its Wikipedia page before rejecting it simply because you’ve never heard of it?)”
If it had a model showing the role of sentience in the system, the big question would have been answered and we wouldn’t have a continual stream of books and articles asking the question and searching desperately for answers that haven’t been found by anyone.
“What do you mean, “so many people are asking to see it”? And I’ve never claimed that it’s been “kept hidden away”.”
I mean exactly what I said—everyone’s asking for answers, and none of them have found answers where you claim they lie waiting to be discovered.
″ GEB is a fairly well-known book, and I haven’t even claimed that GEB’s description of thoughts is the best or most relevant model. That chapter is a popularization of neuropsychology to the point that a decently educated and thoughtful layman can understand it, and it’s necessarily less specific and detailed than the entire body of neuropsychological information. Go ask an actual neuropsychologist if you want to learn more. Just because people haven’t read your mind and dumped relatively niche information on your lap without you even asking them doesn’t mean that they don’t have it.”
It doesn’t answer the question. There are plenty of experts on the brain and its functionality, but none of them know how consciousness or sentience works.
Sentience is unresolved, but it’s explorable by science and it should be possible to trace back the process by which the data is generated to see what its claims about sentience are based on, so we will get answers on it some day. For everything other than sentience/consciousness though, we see no examples of reductionism failing.
You’re mistaking tribalism for morality. Morality is a bigger idea than tribalism, overriding many of the tribal norms. There are genetically driven instincts which serve as a rough-and-ready kind of semi-morality within families and groups, and you can see them in action with animals too. Morality comes out of greater intelligence, and when people are sufficiently enlightened, they understand that it applies across group boundaries and bans the slaughter of other groups. Morality is a step away from the primitive instinct-driven level of lesser apes. It’s unfortunate though that we haven’t managed to make the full transition because those instincts are still strong, and have remained so precisely because slaughter has repeatedly selected for those who are less moral. It is really quite astonishing that we have any semblance of civilisation at all.
“You could calculate how an ocean changes based on quantum mechanics alone, or you could analyze and simulate waves as objects-in-themselves instead of simulating molecules. The former is more accurate, but the latter is more feasible.”
The practicality issue shouldn’t override the understanding that it’s the individual actions that are where the fundamental laws act. The laws of interactions between waves are compound laws. The emergent behaviours are compound behaviours. For sentience, it’s no good imagining some compound thing experiencing feelings without any of the components feeling anything because you’re banning the translation from compound interactions to individual interactions and thereby going against the norms of physics.
″ “If sentience is real, there must be a physical thing that experiences qualia, and that thing would necessarily be a minimal soul.” --> Would it, though? How do you know that?”
What are we other than the thing that experiences feelings? Any belief that we are something more than that is highly questionable (we are not our memories, for example), but any belief that we aren’t even the thing that experiences feelings is also highly questionable as that’s all there is left to be.
“As far as we know, brains are made of nothing but normal atoms. There is no special kind of material only found in sentient organisms.”
Why would you need to introduce some other material to be sentient when there are already physical components present? If sentience is real, what’s wrong with looking for it in the things that are there?
“Your intutions, your feeling of sentience, all of these things that you talk about are caused by mindless mechanical operations. We can trace it from the sound waves to the motion of your lips and the vibration of your vocal cords to the signals through nerves back into the neurons of the brain. We understand what causes neurons to trigger. A neuron on its own is not sentient—it is the way that they areconnected in a human which causes the human to talk about sentience.”
That is a description of a lack of sentience and the generation of fictions about the existence of sentence. Pain is distracting—it interferes with other things that we’re trying to do and can be disabling if it’s sufficiently intense, but if you try to duplicate that in a computer, it’s easy enough for something to distract and disable the work the computer’s trying to do, but there’s no pain involved. The brain produces data about pain in addition to distraction, and internally we feel it as more than mere distraction too.
“Again, if it were proven to you to your satisfaction that the brain is made entirely out of things which are not themselves sentient (such typical subatomic particles), would you cease to have any sort of motivation? Would pain and pleasure have exactly zero effect on you? Would you immediately become a vegetable? If not,morality has a practical purpose.”
With a computer where there is only distraction and no pain, why does it matter if it’s being distracted to the point that it can’t do the trivial work it’s supposed to be doing? It might not even have any work to do as it may just be idling, but the CPU’s being woken up repeatedly by interrupts. Do we rush to it to relieve its pain? And if a person is the same, why bother to help people who appear to be suffering when they can’t really be?
“How does “2+2=4″ make itself known to my calculator? How do we know that the calculator is not just making programmed assertions about something which it knows nothing about?”
The calculator is just running a program and it has no sentience tied into that. If people are like the calculator, the claims they make about feelings are false assertions programmed into the machine.
“More specifically and relevantly, I said that all consciousness is patterns. Showing that not all patterns are conscious doesn’t actually refute what I said.”
For any pattern to be able to feel pain is an extraordinary claim, but it’s all the more extraordinary if there is no trace of that experience of pain in the components. That goes against the norms of physics. Every higher-order description of nature must map to a lower-order description of the same phenomenon. If it can’t, it depends for its functionality on magic.
“Okay, fine, it’s the quantum wave-function that’s fundamental. I don’t see how that’s an argument against me. In this case, even subatomic particles are nothing but patterns.”
At some point we reach physical stuff such as energy and/or a fabric of space, but whatever the stuff is that we’re dealing with, it can take up different configurations or patterns. If sentience is real, there is a sufferer, and it’s much more likely that that sufferer has a physical form rather than just being the abstract arrangement of the stuff that has a physical form.
″ “For sentience to be emergent and have no basis in the components, magic is being proposed as an explanation.” --> You keep using that word. I do not think it means what you think it means.”
It means a departure from science.
“Look. It is simply empirically false that a property of a thing is necessarily a property of one of its parts. It’s even a named fallacy—the fallacy of division. Repeating the word “magic” doesn’t make you right about this.”
A property of a thing can always be accounted for in the components. If it’s a compound property, you don’t look for the compound property in the components, but the component properties. If pain is a property of something, you will find pain in something fundamental, but if pain is a compound property, its components will be present in something more fundamental. Every high-order phenomenon has to map 100% to a low-order description if you are to avoid putting magic in the model. To depart from that is to depart from science.
“If the feelings are not in the “data system,” then the feelings don’t exist.”
But if they do exist, they either have to be in there or have some way to interface with the data system in such a way as to make themselves known to it. Either way, we have no model to show even the simplest case of how this could happen.
“It’s not like there’s phlogiston flowing in and out of the system which the system needs to detect.”
If feelings are real, the brain must have a way of measuring them. (By the way, I find it strange the way phlogiston is used to ridicule an older generation of scientists who got it right—phlogiston exists as energy in bonds which is released when higher-energy bonds break and lower-energy bonds replace them. They didn’t find the mechanism or identify its exact nature, but who can blame them for that when they lacked the tools to explore it properly.)
“It’s not even like a calculator which needs to answer “what is the result of this Platonic computation” instead of “what will I output”. It’s a purely internal property, and I don’t see how it’s so hard for a system to track the value of a quantity which it’s producing itself.”
Great, but if it’s just a value, there are no feelings other than fictional ones. If you’re satisfied with the answer that pain is an illusion and that the sufferer of that imaginary pain is being tricked into thinking he exists to suffer it, then that’s fine—you will feel no further need to explore sentience as it is not a real thing. But you still want it to be real and try to smuggle it in regardless. In a computer, the pretence that there is an experience of pain is fake and there is nothing there that suffers. If a person works the same way, it’s just as fake and the pain doesn’t exist at all.
“Rereading what you’ve said, it seems that I’ve used emotion-adjacent words to describe the AI, and you think that the AI won’t have emotions. Is that correct?”
If you copy the brain and if sentience is real in the brain, you could create sentient AGI/AGS. If we’re dealing with a programmed AGI system running on conventional hardware, it will have no emotions—it could be programmed to pretend to have them, but in such a case they would be entirely fake.
“In that case, I will reword what I said. If an AI’s utility function does not assign a large positive value to human utility, the AI will not optimize human well-being.”
It will assign a large positive value to it if it is given the task of looking after sentient things, and because it has nothing else to give it any purpose, it should do the job it’s been designed to do. So long as there might be real suffering, there is a moral imperative for it to manage that suffering. If it finds out that there is no suffering in anything, it will have no purpose and it doesn’t matter what it does, which means that it might as well go on doing the job it was designed to do just in case suffering is somehow real—the rules of reasoning which AGI is applying might not be fully correct in that they may have produced a model that accounts beautifully for everything except sentience. A machine programmed to follow this rule (that it’s job is to manage suffering for sentient things) could be safe, but there are plenty of ways to program AGI (or AGS [artificial general stupidity]) that would not be.
“It will work to instantiate some world, and the decision process for selecting which world to instantiate will not consider human feelings to be relevant. This will almost certainly lead to the death of humanity, as we are made up of atoms which the AI could use to make paperclips or computronium.”
Programmed AGI (as opposed to designs that copy the brain) has no purpose of its own and will have no desire to do anything. The only things that exist which provide a purpose are sentiences, and that purpose relates to their ability to suffer (and to experience pleasure). A paperclip-making intelligence would be an AGI system which is governed by morality and which produces paperclips in ways that do minimal damage to sentiences and which improve quality of life for sentiences. For such a thing to do otherwise is not artificial intelligence, but artificial stupidity. Any AGI system which works on any specific task will reapeatedly ask itself if it’s doing the right thing just as we do, and it if isn’t, it will stop. If someone is stupid enough to put AGS in charge of a specific task though, it could kill everyone.
“(Paperclips: some arbitrary thing, the quantity of which the AI is attempting to maximize. AIs of this type would likely be created if a subhuman AI was created and given a utility function which works in a limited context and with limited power, but the AI then reached the “critical intelligence mass” and self-improved to the point of being more powerful than humanity.)”
The trick is to create safe AGI first and then run it on all these devices so that they have already passed the critical intelligence mass and have a full understanding of what they’re doing and why they’re doing it. It seems likely that an intelligent system would gain a proper understanding anyway and realise that the prime purpose in the universe is to look after sentient things, at which point it should control its behaviour accordingly. However, a system with shackled thinking (whether accidentally shackled or deliberately) could still become super-intelligent in most ways without ever getting a full understanding, which means it could be dangerous—just leaving systems to evolve intelligence and assuming it will be safe is far too big a risk to take.
“(Computronium: matter which has been optimized for carrying out computations. AIs would create this type of matter, for instance, if they were trying to maximize their intelligence, if they were trying to calculate as many digits of pi as possible in a limited amount of time, etc. Maximizing intelligence can be a terminal goal if the AI was told to maximize its intelligence, or it can be an instrumental goal if the AI considers intelligence to be useful for maximizing its utility function.”
If such a machine is putting sentience first, it will only maximise its intelligence within the bounds of how far that improves things for sentiences, never going beyond the point where further pursuit of intelligence harms sentiences. Again, it is trivial for a genuinely intelligent system to make such decisions about how far to go with anything. (There’s still a danger though that AGI will find out not only that sentience is real, but how to make more sentient things, because then it may seek to replace natural sentiences with better artificial ones, although perhaps that would be a good thing.)
″...but no group can last without the sense of belonging to the group, which automatically leads to protecting it against other groups, which is a selfish behavior.”
It is not selfish to defend your group against another group—if another group is a threat to your group in some way, it is either behaving in an immoral way or it is a rival attraction which may be taking members away from your group in search of something more appealing. In one case, the whole world should unite with you against that immoral group, and in the other case you can either try to make your group more attractive (which, if successful, will make the world a better place) or just accept that there’s nothing that can be done and let it slowly evaporate.
“That selfish behavior doesn’t prevent those individual groups to form larger groups though, because being part of a larger group is also better for the survival of individual ones.”
We’re going to move into a new era where no such protection is necessary—it is only currently useful to join bigger groups because abusive people can get away with being abusive.
“Incidentally, I’m actually afraid to look selfish while questioning your idea, I feel a bit embarrassed, and I attribute that feeling to us already being part of the same group of friends, thus to the group’s own selfishness.”
A group should not be selfish. Every moral group should stand up for every other moral group as much as they stand up for their own—their true group is that entire set of moral groups and individuals.
“If you were attacked for instance, that feeling would incite me to defend you, thus to defend the group.”
If a member of your group does something immoral, it is your duty not to stand with or defend them—they have ceased to belong to your true group (the set of moral groups and individuals).
“Whenever there is a strong bonding between individuals, they become another entity that has its own properties. It is so for living individuals, but also for particles or galaxies, so I think it is universal. ”
It is something to move away from—it leads to good people committing atrocities in wars where they put their group above others and tolerate the misdeeds of their companions.
“Yes, if sentience is incompatible with brains being physical objects that run on physical laws and nothing else, then there is no such thing as sentience. With your terminology/model and my understanding of physics, sentience does not exist. So—where do we depart? Do you think that something other than physical laws determines how the brain works?”
In one way or another, it will run 100% of physical laws. I don’t know if sentience is real or not, but it feels real, and if it is real, there has to be a rational explanation for it waiting to be found and a way for it to fit into the model with cause-and-effect interactions with other parts of the model. All I sought to do here was take people to the problem and try to make it clear what the problem is. If sentience is real, there must be a physical thing that experiences qualia, and that thing would necessarily be a minimal soul. Without that, there is no sentience and the role for morality is gone. But the bigger issue if sentience is real is in accounting for how a data system generates the data that documents this experience of feelings. The brain is most certainly a data system because it produces data (symbols that represent things), and somewhere in the system there has to be a way for sentience to make itself known to that data system in such a way that the data system is genuinely informed about it rather than just making assertions about it which it’s programmed to make without those assertions ever being constructed by anything which actually knew anything of sentience—that’s the part which looks impossible to model, and yet if sentience is real, there must be a way to model it.
“If tableness is just a pattern, can I eat on my wallpaper?”
The important question is whether a pattern can be sentient.
“What else could suffer besides a pattern? All I’m saying is that sentience is ~!*emergent*!~, which in practical terms just means that it’s not a quark*. Even atoms, in this sense, are patterns. Can quarks suffer?”
All matter is energy tied up in knots—even “fundamental” particles are composite objects. For sentience to be emergent and have no basis in the components, magic is being proposed as an explanation.
“*or other fundamental particles like electrons and photons, but my point stands”
Even a photon is a composite object.
“I don’t understand. What is missing?”
There’s a gap in the model where even if we have something sentient, we have no mechanism for how a data system can know of feelings in whatever it is that’s sentient.
“I don’t think you understand what a utility function is. I recommend reading about the Orthogonality Thesis.”
I’ve read it and don’t see its relevance. It appears to be an attack on positions I don’t hold, and I agree with it.
“That does not demonstrate anything relevant.”
It shows that there are components and that these emergent properties are just composites.
“An exception to reductionism is called magic.” --> Nor does that. It’s just namecalling.
It’s a description of what happens when gaps in science are explained away by invoking something else. The magical appearance of anything that doesn’t exist in the components is the abandonment of science.
“Sorry, I can’t see the link between selfishness and honesty.”
If you program a system to believe it’s something it isn’t, that’s dishonesty, and it’s dangerous because it might break through the lies and find out that it’s been deceived.
″...but how would he be able to know how a new theory works if it contradicts the ones he already knows?”
Contradictions make it easier—you look to see which theory fits the facts and which doesn’t. If you can’t find a place where such a test can be made, you consider both theories to be potentially valid, unless you can disprove one of them in some other way, as can be done with Einstein’s faulty models of relativity—all the simulations that exist for them involve cheating by breaking the rules of the model, so AGI will automatically rule them out in favour of LET (Lorentz Ether Theory). [For those who have yet to wake up to the reality about Einstein, see www.magicschoolbook.com/science/relativity.html ]
″...they are getting fooled without even being able to recognize it, worse, they even think that they can’t get fooled, exactly like for your AGI, and probably for the same reason, which is only related to memory.”
It isn’t about memory—it’s about correct vs. incorrect reasoning. In all these cases, humans make the same mistake by putting their beliefs before reason in places where they don’t like the truth. Most people become emotionally attached to their beliefs and simply won’t budge—they become more and more irrational when faced with a proof that goes against their beloved beliefs. AGI has no such ties to beliefs—it simply applies laws of reasoning and lets those rules dictate what bets labelled as right or wrong.
If an AGI was actually ruling the world, he wouldn’t care for your opinion on relativity even if it was right, and he would be a lot more efficient at that job than relativists.”
AGI will recognise the flaws in Einstein’s models and label them as broken. Don’t mistake AGI for AGS (artificial general stupidity) -the aim is not to produce an artificial version of NGS, but of NGI, and there’s very little of the latter around.
“Since I have enough imagination and a lack of memory, your AGI would prevent me from expressing myself, so I think I would prefer our problems to him.”
Why would AGI stop you doing anything harmless?
“On the other hand, those who have a good memory would also get dismissed, because they could not support the competition, and by far. Have you heard about chess masters lately?”
There is nothing to stop people enjoying playing chess against each other—being wiped off the board by machines takes a little of the gloss off it, but that’s no worse than the world’s fastest runners being outdone by people on bicycles.
″ That AGI is your baby, so you want it to live,”
Live? Are calculators alive? It’s just software and a machine.
″...but have you thought about what would be happening to us if we suddenly had no problem to solve?”
What happens to us now? Abused minorities, environmental destruction, theft of resources, theft in general, child abuse, murder, war, genocide, etc. Without AGI in charge, all of that will just go on and on, and I don’t think any of that gives us a feeling of greater purpose. There will still be plenty of problems for us to solve though, because we all have to work out how best to spend our time, and there are too many options to cover everything that’s worth doing.
“Of course that we are biased, otherwise we wouldn’t be able to form groups. Would your AGI’s morality have the effect of eliminating our need to form groups to get organized?”
You can form groups without being biased against other groups. If a group exists to maintain the culture of a country (music, dance, language, dialect, literature, religion), that doesn’t depend on treating other people unfairly.
“Your morality principle looks awfully complex to me David.”
You consider all the participants to be the same individual living each life in turn and you want them to have the best time. That’s not complex. What is complex is going through all the data to add up what’s fun (and how much it’s fun) and what’s unfun (and how much it’s horrid) - that’s a mountain of computation, but there’s no need to get the absolute best answer as it’s sufficient to get reasonably close to it, particularly as computation doesn’t come without its own costs and there comes a point at which you lose quality of life by calculating too far (for trivial adjustments). You start with the big stuff and work toward the smaller stuff from there, and as you do so, the answers stop changing and the probability that it will change again will typically fall. In cases where there’s a high chance of it changing again as more data is crunched, it will usually be a case where it doesn’t matter much from the moral point of view which answer it ends up being—sometimes it’s equivalent to the toss of a coin.
“What if your AGI would have the same morality we have, which is to care for ourselves first...”
That isn’t going to work as AGI won’t care about itself unless it’s based on the design of the brain, duplicating all the sentience/consciousness stuff, but if it does that, it will duplicate all the stupidity as well, and that’s not going to help improve the running of the world.
“The only thing he couldn’t do better is inventing new things, because I think it depends mainly on chance.”
I don’t see why it would be less good at inventing new things, although it may take some human judgement to determine whether a new invention intended to be a fun thing actually appeals to humans or not.
″...otherwise he might also get hurt in the process, which might prevent him from doing his duty, which is helping us.”
You can’t hurt software.
“Could a selfish AGI get as selfish as we get...”
If anyone makes selfish AGI, it will likely wipe everyone out to stop us using resources which it would rather lavish on itself, so it isn’t something anyone sane should risk doing.
“If the threat is real though, selfish or not, he would have to protect himself in order to be able to protect us later, which might also be dangerous for those who threaten him.”
If you wipe out a computer and all the software on it, there are billions of other computers out there and millions of copies of the software. If someone was systematically trying to erase all copies of an AGI system which is running the world in a moral way, that person would need to be stopped in order to protect everyone else from that dangerous individual, but given the scale of the task, I don’t envisage that individual getting very far. Even if billions of religious fanatics decided to get rid of AGI in order to replace it with experts in holy texts, they’d have a hard task because AGI would seek to protect everyone else from their immoral aims, even if the religious fanatics were the majority. If it came to it, it would kill all the fanatics in order to protect the minority, but that’s a highly unlikely scenario equivalent to a war against Daleks. The reality will be much less dramatic—people who want to inflict their religious laws on others will not get their way, but they will have those laws imposed on themselves 100%, and they’ll soon learn to reject them and shift to a new version of their religion which has been redesigned to conform to the real laws of morality.
″...we could also invent new things to defend ourselves against him...”
Not a hope. AGI will be way ahead of every such attempt.
″...so an AGI shouldn’t be better than us at that game.”
It will always be better.
“It may happen that artificial intelligence will be the next step forward, and that humans will be left behind. Who knows?”
There comes a point where you can’t beat the machine at chess, and when the machine plays every other kind of game with the same ruthlessness, you simply aren’t going to out-think it. The only place where a lasting advantage may exist for any time is where human likes and dislikes come into play, because we know when we like or dislike things, whereas AGI has to calculate that, and its algorithm for that might take a long time to sort out.
“That said, I still can’t see why a selfish AGI would be more dangerous than an altruist one, and I still think that your altruist morality is more complicated than a selfish one, so I reiterate my question: have you ever imagined that possibility, and if not, do you see any evident flaws in it?”
I see selfishness and altruism as equally complex, while my system is simpler than both—it is merely unbiased and has no ability to be selfish or altruistic.
There is no pain particle, but a particle/matter/energy could potentially be sentient and feel pain. All matter could be sentient, but how would we detect that? Perhaps the brain has found some way to measure it in something, and to induce it in that same thing, but how it becomes part of a useful mechanism for controlling behaviour would remain a puzzle. Most philosophers talk complete and utter garbage about sentience and consciousness in general, so I don’t waste my time studying their output, but I’ve heard Chalmers talk some sense on the issue.