Eliezer, didn’t you say that humans weren’t designed as optimizers? That we satisfice. The reaction you got is probably a reflection of that. The scenario ticks most of the boxes humans have, existence, self-determination, happiness and meaningful goals. The paper clipper scenario ticks none. It makes complete sense for a satisficer to pick it instead of annihilation. I would expect that some people would even be satisfied by a singularity scenario that kept death as long as it removed the chance of existential risk.
Will_Pearson
Is this a maths question or a reasoning question?
What would the cult have said if the man had said something a long the lines of this to the first question.
“As probability is a subjective quality which must take all the information available to me to be valid, I’m not going to take the assumptions that were given to me. From the my estimate of your bodily proportions and the distribution of the height and weight of the general populace, and your low tone of voice compared to the average I would give you a 90% chance of being male.”
Once there were two aliens Phlix and Claz, deep in the galactic cluster.
Phlix: I’ve figured out how to travel between stars, all we have to do is create a machine that can improve its speed by 10% in a second. Claz: So if it is going 10km/h it can go 11 km/h… that is not going to help us reach the nearest stars. Phlix: You are neglecting the power of the exponent, if it continuously increases its speed by 10% every second, after an hour we would be travelling at 8 10^15 km/h or 2.2 10^16 m/s. Claz: But we don’t understand energy and motion yet. Perhaps you should formulate a theory of these, to decide the feasibility of what you want to do first, before you set your heart on building this machine.. Phlix: But that would be admitting the possibility of defeat and not being able to create the future I want, I need to maintain my faith, else I might give up prematurely, without having done all that I could do. The future of our race depends on it, no longer would we be tied to the fate of a single planet. It is only a matter of time before a star goes nova in our vicinity, the geological records have shown the number of time that life on this planet has been reduced to less complex life, due to the radiation bursts. Perhaps next time we won’t be so lucky and the entire atmosphere will be ionised. Claz: If this speed of travel is so easy, why haven’t we seen other intelligent life forms visit us, from all across the skies yet. Phlix: Well that must be because intelligent life forms are very rare.
I read fantasy, though less so now, mainly because it is groups of people banding together to achieve a goal they knew was just or worthwhile (generally saving the world, defeating the evil forces). The actual magic was just a spice that leant an air of mystery, and unpredictability (so I am more a fan of George Martin, David Gemmell and Guy Gavriel Kay rather than Raymond E. Feist and David Eddings. Robert Jordan lost me when the good guys split up into bickering factions).
I’m just disappointed that AI is at the herding cats stage (myself included), when trying achieve consensus on how to actually proceed. No party of merry adventurers are we.
With regards to the allure of magic compared to science, magic tends to need a lot less in terms of support mechanisms. The inability to get precisely machined parts would hamper your attempts to build a transportation device, where magic, depending upon the sub-genre, can do just about anything with anything, so is much more empowering and self-sufficient.
His polemical style reminds me too much of political rhetoric for my liking. It hooks too much into the us vs them psychology e.g. rationalists vs irrationalists.
“Finally, I am going to take a strictly realist perspective on quantum mechanics—the quantum world is really out there, our equations describe the territory and not our maps of it,”
Why isn’t this an example of the mind projection fallacy? I know you said to give you a break, but I really don’t like it when people contradict themselves.
“Right now, we’ve got the worst of both worlds. Science isn’t really free, because the courses are expensive and the textbooks are expensive. But the public thinks that anyone is allowed to know, so it must not be important.”
Anyone is allowed to pick up and read a bible. They are even given away free! The public still seems to rate the teachings in that though.
If I was trying to spread science, I wouldn’t make it scare, I would make it social. How about a roleplaying game, Scientist: The Discovery! Scenarios are scientific problems with real world data and the players level up by solving them. Or perhaps fantastical, such as deflecting asteroids, but using real equations.
Also cults are not the thing you have to get science to the level of, today it is celebrity worship/sports following/world of warcraft.
The whole chanting thing put me off religion, I’d much prefer a ritual dance. And no bloody sacrifices.
I’m curious what exactly your HR problems are with SIAI, it doesn’t seem to have any jobs or research posts open.
All competitive situations against ideal learning agents are anti inductive in this sense. Because they can note regularities in their actions and avoid them in the future as well as you can note regularities in their actions and exploit them. The usefulness of induction is based on the relative speeds of the induction of the learning agents.
As such anti induction appears in situations like bacterial resistance to antibiotics. We spot a chink in the bacterias armour, and we can predict that that chink will become less prevalent and our strategy less useful.
So I wouldn’t mark markets as special, just the most extreme example.
How do Ebborians get different names when they fission?
Eliezer or anyone else: I am puzzled why the mirror itself doesn’t act as a sensitive thingy. A mirror that deflects a photon gains some momentum (hence solar sails or whatever), so I’d expect the configurations to be
“Photon from A to B; and mirror at A with momentum X+e.” (0 + -i) “Photon from A to C; and mirror at A with momentum X.” (-1 + 0i)
“You cannot write a Bayes program that inputs all the right data and generates garbage.”
That sounds like a challenge. Care to formalise things so this theory can be tested?
There hasn’t been much evidence of atheists forming groups that have the positive aspects that a church/synagogue/mosque holds in the social life of some humans. So you might forgive a theist pretending to be a rationalist, for not holding the probability of this happening very high, and that the world would lack said institutions and would be a worse place.
If rationalists truly wants to get rid of religions, without getting rid of humans, we would have to ask ourselves, “What do humans get out of being part of a religion?” And then provide that through organisations.
And please no strawmen of the comfort of ignorance, I am talking about reassurance of being with people who are trying to hold the same goal system.
I don’t really have a good enough grasp on the world to predict what is possible, it all seems to unreal.
One possibility is to jump one star away back towards earth and then blow up that star, if that is the only link to the new star.
The thing that bothers me about quantum physics is that I would like to do total simulations of things using universal laws, all at the same level. This is how I have come to understand things. So I want to model the half-silvered mirror quantumly and the detectors as well.
It is deeply disturbing to have half silvered mirrors perform operations on things, with no way of determining what happened to them in return, because they are not modelled at the same level, but at a higher level.
This goes back to my problem with the conservation of momentum on half-silvered mirrors. Because I figured out that the photon that deflects should red shift (as it transfers momentum to the mirror), and so shouldn’t be identical. I find myself wanting to talk about the difference between what is tracked by physics and what we know. The very-very slightly red shifted photon may not be detectable by us, but should after 50 billion times of being deflected in the same way be different. That is the physics has to keep track of things we can’t discern very small amounts of.
Oh for the time to learn all this properly.
Most of my imaginations about QM not being complete are to do with exotic conditions (start/end of universe) revealing non identical properties normally hidden.
With international affairs, isn’t stopping the aggression the main priority? That is stopping the death and suffering of humans on both sides? Sure it would be good to punish the aggressors rather than the retaliators but if that doesn’t stop the fighting it just means more people are dying.
Also there is a difference between the adult and the child, the adult relies on the law of the land for retaliation the child takes it upon himself when he continues the fight. That is the child is a vigilante, and he may punish disproportionately e.g. breaking a leg for a dead leg.
“Except to remark on how many different things must be known to constrain the final answer.”
What would you estimate the probability of each thing being correct is?
Don’t you need a person predicate as well? If the RPOP is going to upload us all or something similar, doesn’t ve need to be sure that the uploads will still be people.
There are some types of knowledge that seem hard to come by (especially for singletons). The type of knowledge is knowing what destroys you. As all knowledge is just an imperfect map, there are some things a priori that you need to know to avoid. The archetypal example is in-built fear of snakes in humans/primates. If we hadn’t had this while it was important we would have experimented with snakes the same way we experiment with stones/twigs etc and generally gotten ourselves killed. In a social system you can see what destroys other things like you, but the knowledge of what can kill you is still hard won.
If you don’t have this type of knowledge you may step into an unsafe region, and it doesn’t matter how much processing power or how much you correctly use your previous data. Examples that might threaten singletons:
1) Physics experiments, the model says you should be okay but you don’t trust your model under these circumstances, which is the reason to do the experiment. 2) Self-change, your model says that the change will be better but the model is wrong. It disables the system to a state it can’t recover from, i.e. not an obvious error but something that renders it ineffectual. 3) Physical self-change. Large scale unexpected effects from feedback loops at a different levels of analysis, e.g. things like the swinging/vibrating bridge problem, but deadly.
Eliezer, I’m not quite sure why you are obsessed with the ability to find mathematical truths. You can different sets of mathematical truths if you take the parallel postulate true or not. Which ones are applicable in the real world depends where you are.
Unless you know where you are in the universe can not know what is friendly for you to do (or even useful).
Evolution less complex than a creature?
Lets break it down Variation—Even including all the things you don’t mention such as variable mutation rates across the length of the genome, transposons, hox genes and segmented body types, your picture is still probably right
Selection—Since what selects which creature dies or reproduces is the whole rest of the ecosystem and physical surroundings, I would respectfully disagree that this is simpler than the creature.
Simple things cannot drive the complexity of things very high. Have a look at the simple environments of Tierra and other Alife models and see how they tend to peter out without anything very much of interest being created. Despite using the “same” variation and selection of evolution.
My main problem with your view of the human brain, is that the brain does not remain untouched while it learns. It doesn’t do what you want to do, but still it isn’t unchanged. I think it modifies itself down to the equivalent of machine code, it is just invisible to the conscious brain it is all being done lower down. There is no reason the conscious monkey part of the brain should have permission to alter the low level bits, if it doesn’t understand anything about it. Nothing else could explain to me how it learns echolocation in the blind, repurposes the visual cortex for braille and expands the hippocampus of cabbies. Tell me what exactly is the protected structure of the brain that doesn’t or can’t change, because I just don’t see it.
There might be bits protected from the top level, but then the top conscious level has no clue on how to alter it, and possibly never could considering the complexity differential between the low levels and the top levels, and the difficulties at altering a moving target. This does not mean they are not altered.
There were three men on a sinking boat.
The first said, “We need to start patching the boat else we are going to drown. We should all bail and patch.”
The second said, “We will run out of water in ten days, if we don’t make land fall. We need to man the rigging and plot a course.”
The third said, “We should try and build a more sea worthy ship. One that wasn’t leaking and had more room for provisions, then we wouldn’t have had this problem in the first place. It also needs to be giant squid proof.”
All three views are useful, however the amount of work that we need on each is dependent on their respective possibility. As far as I am concerned the world doesn’t have enough people working on the second view.