Or more concisely: sharp distinctions regarding fuzzy concepts are meaningless.
Joseph_Knecht
Karma: 41
Optimistically, I would say that if the murderer perfectly knew all the relevant facts, including the victim’s experience, ve wouldn’t do it
The murderer may have all the facts, understand exactly what ve is doing and what the experience of the other will be, and just decide that ve doesn’t care. Which fact is ve not aware of? Ve may understand all the pain and suffering it will cause, ve may understand that ve is wiping out a future for the other person and doing something that ve would prefer not to be on the receiving end of, may realize that it is behavior that if universalized would destroy society, may realize that it lessens the sum total of happiness or whatever else, may even know that “ve should feel compelled not to murder” etc. But at the end of the day, ve still might say, “regardless of all that, I don’t care, and this is what I want to do and what I will do”.
There is a conflict of desire (and of values) here, not a difference of fact. Having all the facts is one thing. Caring about the facts is something altogether different.
--
On the question of the bedrock of fairness, at the end of the day it seems to me that one of the two scenarios will occur:
(1) all parties happen to agree on what the bedrock is, or they are able to come to an agreement.
(2) all parties cannot agree on what the bedrock is. The matter is resolved by force with some party or coalition of parties saying “this is our bedrock, and we will punish you if you do not obey it”.
And the universe itself doesn’t care one way or the other.
Caledonian: Every mathematical statement is a claim about the behavior of the physical world.
Please interpret the following statements for me in terms of the behavior of the physical world, and tell me which branch of physics deals with the behavior of each:
The cardinality of the set of real numbers is greater than the cardinality of the set of natural numbers.
The continuum hypothesis is independent of ZF and ZFC set theory.
There are no solutions to the equation a^n + b^n = c^n for non-zero integers a, b, and c and integer n > 2.
Caledonian, the childish “I have a secret that I’m not going to tell you, but here’s a hint” bs is very annoying and discourages interacting with you. If you’re not willing to spell it out, just don’t say it in the first place. Nobody cares to play guessing games with you.
Thanks for the explanations, Bob.
Bob: The point of using 3^^^3 is to avoid the need to assign precise values… Once you accept the premise that A is less than B (with both being finite and nonzero), you need to accept that there exists some number k where kA is greater than B.
This still requires that they are commensurable though, which is what seeking a strong argument for. Saying that 3^^^3 dust specks in 3^^^3 eyes is greater harm than 50 years of torture means that they are commensurable and that whatever the utilities are, 3^^^3 specks divided by 50 years of torture is greater than 1.0. I don’t see that they are commensurable. A < B < C < D doesn’t imply that there’s some k such that kA>D.
Consider: I prefer Bach to Radiohead (though I love both). That doesn’t imply that there’s some ratio of Bach to Radiohead, or that I think a certain number of Radiohead songs are collectively better than or more desirable than, for example, the d-minor partita. Even if I did in some cases believe that 10 Radiohead songs were worth 1 Bach prelude and fugue, that would just be my subjective feeling. I don’t see why there must be an objective ratio, and I can’t see grounds for saying what such a ratio would be. Likewise for dust-specks and torture.
Like Mitchell, I would like to see exactly how people propose to assign these ratios such that a certan number of one harm is greater than a radically different harm.
This post makes a valuable point, but the point is weakened by too much hyperbole—or rather by hyberbole that seems like a plausible non-hyperbolic statement that the writer might actually believe.
Whenever I hear someone describe quantum physics as “weird”—whenever I hear someone bewailing the mysterious effects of observation on the observed, or the bizarre existence of nonlocal correlations, or the incredible impossibility of knowing position and momentum at the same time—then I think to myself: This person will never understand physics no matter how many books they read.
I take the last clause (“This person will never understand physics no matter how many books they read”) to mean “will never understand physics no matter what they do”, since nobody seriously thinks you can really understand physics by just reading books, and there is no special relation between books and the other point being made, so I take that as evidence that ‘books’ is incidental and not intrinsic to the point.
If that is the case, then Eliezer would be committed to the idea that Einstein and Feynman, no matter how long they lived, would not be capable of understanding physics. Which is absurd! Yes, Einstein had intuitions that he found very hard to give up; yes, Feynman was limited to the theory of his day; but you still cannot mean that they would not have ever been able to understand physics, no matter how long they lived and what they did.
Surprise and weirdness are not qualities of the world but of model-making monkeys in the world. This is a valuable point. And thank you for it.
In the video link that komponisto gave, the relevant section of the video starts at 49:00 or so. He doesn’t argue there though that consciousness—he uses the term sentience—will necessarily remain a mystery but only that this might turn out to be the case. He makes the analogy of trying to think about time before the big bang and that there is some kind of conceptual category-type error going on there that gives it its subjective feeling of being mysterious and unknowable, and states that this may be the case with sentience/consciousness, free-will, etc.
Caledonian, that’s mere sophistry to say “mathematics is physics because it is performed by a brain or analogous physical device”.
According to that definition, no matter what you study at university, you are really doing physics. Every single human being that has ever earned a university degree earned a physics degree (since English is Physics, Art History is Physics, etc.), and every individual whose work involves use of her brain (even if only for respiration and basic metabolic processes) is a physicist.
I think I’ll stick with the understanding of physics that the rest of the world uses.
I think the shot of adrenaline to the ego is what gives the sense of high in most cases, and what motivates most scientists. And it probably is almost entirely the source of the high of the non-world changing and minor discoveries.
Having said that, I do think that in some cases, very few, there is perhaps a stronger element of what Eliezer briefly touched on towards the end of the essay: that one has just added to the sum total of humanity’s knowledge, and inched us toward the perfect understanding of the world around us that science constantly seeks.
To think that one has just discovered something that will affect all of humanity for the rest of time by adding to the knowledge we have and providing a foundation for all knowledge that builds upon it is a dizzying thought, and I think the high is not only that “I’m such a genius and I’ll be remembered forever and be envied by all my contemporaries”, but also consists in the realization of the incalculable consequences of what you have just discovered.
Of course, this applies to discoveries of the nature of Newton or Darwin, rather than lesser discoveries, and I’d attribute the high of lesser discoveries to more egocentric thoughts. (And perhaps in Newton’s case as well, since he was an quite a self-centered individual, but that’s another subject.)
To summarize, while it may be that the ego-centered explanations of the high is the dominant explanation in all minor or trivial discoveries, and is present in all greater discoveries, in some cases, the high may be even more strongly based on the sense of steering the future of mankind, or at least science, of leading us into new territory. If it feels good to help an old lady across the street, how would it feel to give a gift to the trillions of human beings that do not yet exist? And this explains why the high is probably that much greater—at least upon reflection—for something that one thinks might not have been discovered for a long time otherwise, as opposed to the things that were in the air at the time and would certainly have been discovered in the very near future by somebody else (e.g., Archimedes’ method of exhaustion [if he’d have sensed the implications], close to the modern use of limits in calculus and analysis, versus Watson/Crick who were racing to beat Linus Pauling).
mtraven: many of the posters in this thread—myself included—have said that they don’t believe in morality (meaning morality and not “values” or “motivation”), and yet I very highly doubt that many of us are clinically psychopaths.
Not believing in morality does not mean doing what those who believe in morality consider to be immoral. Psychopathy is not “not believing in morality”: it entails certain kinds of behaviors, which naive analyses of attribute to “lack of morality”, but which I would argue are a result of aberrant preferences that manifest as aberrant behavior and can be explained without recourse to the concept of morality.
Like many others here, I don’t believe that there is anything like a moral truth that exists independently of thinking beings (or even dependently on thinking beings in anything like an objective sense), so I already live in something like that hypothetical. Thus my behavior would not be altered in the slightest.
@Jagadul:
by “constraints”, I meant that Eliezer specified only that some particular processes happening in the brain are sufficient for choice occurring, which my example refuted, to which you added the ideas that it is not mere happening in the brain but also the additional constraints entailed by concepts of Eliezer-the-person and body-shell-of-Eliezer and that the former can be destroyed while the latter remains, which changes ownership of the choice, etc.
Anyway, I understand what you’re saying about choice as a higher-level convenience term, but I don’t think it is helpful. I think it is a net negative and that we’d do better to drop it. You gave the thought, “given these options, what will he choose?”, but I think the notion of choice adds nothing of value to the similar question, “given these options, what will he do?” You might say that it is different, since a choice can be made without an action occurring, but then I think we’d do better to say not “what will he choose?” but something more like “what will he think?”, or perhaps something else depending on the specifics of the situation under consideration.
I believe there’s always a way of rephrasing such things so as not to invoke choice, and all that we really give up is the ability to talk about totally generic hypothetical situations (where it isn’t specified what the “choice” is about). Whenever you flesh out the scenario by specifying the details of the “choice”, then you can easily talk about it more accurately by sidestepping the notion of choice altogether.
I don’t think that “choice” is analogous to Newtonian mechanics before relativity. It’s more akin to “soul”, which we could have redefined and retrofitted in terms of deterministic physical processes in the brain. But just as it makes more sense to forget about the notion of a soul, I think it makes more sense to forget about that of “choice”. Just as “soul” is too strongly associated with ideas such as dualism and various religious ideas, “choice” is too strongly associated with ideas such as non-determinism and moral responsibility (relative to some objective standard of morality). Instead of saying “I thought about whether to do X, Y, or Z, then choose to do X, and then did X”, we can just say “I thought about whether to do X, Y, or Z, then did X.”
@Constant:
I think “choice” is closer to “caloric” than “heat”, because I don’t believe there is any observable mundane phenomenon that it refers to. What do you have in mind that cannot be explained perfectly well without supposing that a “choice” must occur at some point in order to explain the observed phenomenon?
Personally, I don’t there is a trick, and I don’t think he’s keeping it private for those reasons. I think his method, if something so obvious (which is not to say easy) can be called a method, is to discuss the issue and interact with the person long enough to build up a model of the person, what he values and fears most, and then probe for weaknesses & biases where that individual seems most susceptible, and follow those weaknesses—again and again.
I think most, perhaps all, of us, unless we put our fingers in our ears and refuse to honestly engage, are capable of being convinced by a skilled interlocutor who has MUCH more experience thinking about the issue than we do.
Of course, I could be wrong, and there could be some argument that would convince me in minutes, or there could be some trick, but I’d be very surprised if so.
@DaveInNYC: what you can and can’t assume is not relevant to whether the transcripts should be private or not. If they were public, anybody predisposed to explanations like “they must have been more simple-minded than me” could just as easily find another equally “compelling” explanation, like “I didn’t think of that ‘trick’, but now that I know it, I’m certain I couldn’t be convinced!”
I personally think they should remain private, as frustrating as it is to not know how Eliezer convinced them. Not knowing how Eliezer did it nicely mirrors the reality of our not knowing how a much smarter AGI might go about it.
Eliezer: if you’re going to point to the AI Box page, shouldn’t you update it to include more recent experiments (like the ones from 2005 where the gatekeeper did not let the AI out)?
When did “genius” (as in “just another Jewish genius”) as a term become acceptable to use in the sense of mere “exceptional ability” without regard to accomplishment/influence or after-the-fact eminence? I know it is commonly (mis-)used in this sense, but it seems to me that “unaccomplished genius” should be an oxymoron, and I’m somewhat surprised to see it used in this sense so much in this thread (and on this forum).
I have always considered the term to refer (after the fact) to those individuals who shaped the intellectual course of humanity—e.g., Shakespeare, Newton, Darwin, Einstein—and not just high-IQ individuals who may or may not actually do anything of consequence. It is what Newton and Mozart and Picasso actually did, the effect they had on intellectual history, that justifies our calling them geniuses, not the mere fact that they were exceptionally talented.
What do others think? Perhaps we misuse the word because there is no other single word that is appropriate? Or is there some word I’m not thinking of to describe exceptionally intelligent and creative people (without regard to what they do with their abilities)? “Brilliant” as an adjective, if pronounced emphatically enough, conveys the sense, but it’s not a noun.
Eliezer: no comment on my point that ‘single-levelness’ is an attribute of your model of reality rather than of reality itself? And that saying “reality is single-level” is therefore misleading.
The mathematics of physics is just an infinitesimal part of all of mathematics.
Physics could at some point be completely solved, which is to say that at some point, there would be no further knowledge that would ever allow us to do anything new, to make any better a prediction, to do anything more efficiently, etc.
There is no such limit to mathematics though, because mathematics, unlike physics, is not constrained by reality. It only needs to be self consistent (under perhaps limitless different conceptions of consistency) given a particular starting point, and there is no limit to the number of starting points or perspectives upon which can be built new systems. And there are concepts analogous to quantity, transformation, shape, etc., that that have not yet been invented but will be fertile ground for new branches of mathematics someday.
I think it’s possible that all useful mathematics could someday be discovered (if you consider all art useless), but that would still be just an infinitesimal part of all of mathematics. To say that mathematics could be exhausted is to say that all stories may one day have been told, and fiction may be exhausted. It just can’t happen.
Caledonian: you said “Your visualizations include such details? As the description didn’t include such details, they’re necessarily undefined—so why did you define them out of their uncertainty?”
I understood from your statement that you expressed surprise that the reported visualization contained such details as “which side of the street the person is walking down”. This implied to me that you believe it is possible to visualize a man walking down a street, but not be either walking down the left or right side or in the street itself, etc.
Ian C: One advantage of the Two Party Swindle is that swing-voters usually decide an election. That is, the small percent of people who don’t fall for Us vs. Them.
That’s small consolation when what you actually get to decide is whether you’ll receive a hefty kick to the groin or a stick in the eye.