One of the most obvious examples of commonly encountered unreliable information are advertisements. Gilbert’s results suggest that knowing that the information in advertisements is highly unreliable doesn’t make you immune to their effects. This suggests that it’s a good idea to avoid perceiving advertisements entirely, especially in situations where you’re trying to concentrate on something else. The obvious way to do this is to aggressively use ad-blockers wherever possible; unfortunately there are still media where this isn’t practical.
Sebastian_Hagen2
Karma: 185
“A short time?” Jeffreyssai said incredulously. “How many minutes in thirty days? Hiriwa?”
“28800, sensei,” she answered. “If you assume sixteen-hour waking periods and daily sleep, then 19200 minutes.”I would have expected the answers to be 43200 (30d 24h/d 60/h) and 28800 (30d 16h/d 60/h), respectively. Do these people use another system for specifying time? It works out correctly if their hours have 40 minutes each.
Aside from that, this is an extremely insightful and quote-worthy post. I have^W^W My idiotic past-selves had a bad tendency to cognitively slow down in the absence of interesting and time-critical problems to solve. Accordingly, I find the hints about how to debug those tendencies very interesting. I find it rather quaint that those people still spend a significant part of their time sleeping, however.
To make it clear why you would sometimes want to think about implied invisibles, suppose you’re going to launch a spaceship, at nearly the speed of light, toward a faraway supercluster. By the time the spaceship gets there and sets up a colony, the universe’s expansion will have accelerated too much for them to ever send a message back.
Ah! Now I see that my earlier claim about sane utility functions not valuing things that couldn’t be measured even in principle was obviously bogus. Some commentors poked holes in the idea before, but a number of issues complicating the p-zombie case prevented me from seeing how big those were. This example made it really clear to me. Thank you!
J Thomas: You’re neglecting that there might be some positive-side effects for a small fraction of the people affected by the dust specks; in fact, there is some precedent for this. The resulting average effect is hard to estimate, but (considering that dust specks seem to mostly add entropy to the thought processes of the affected persons), would likely still be negative.
Copying g’s assumption that higher-order effects should be neglected, I’d take the torture. For each of the 3^^^3 persons, the choice looks as follows:
1.) A 1/(3^^^3) chance of being tortured for 50 years. 2.) A 1 chance of getting a dust speck.
I’d definitely prefer the former. That probability is so close to zero that it vastly outweighs the differences in disutility.
This post reminds me a lot of DialogueOnFriendliness.
There’s at least one more trivial mistake in this post:
Is their nothing more to the universe than their conflict?
s/their/there/Constant wrote:
Arguably the difficulty the three have in coming to a conclusion is related to the fact that none of the three has anything close to a legitimate claim on the pie.
If you modify the scenario by postulating that the pie is accompanied by a note reading “I hereby leave this pie as a gift to whomever finds it. Enjoy. -- Flying Pie-Baking Monster”, how does that make the problem any easier?
They only depend to within a constant factor. That’s not the problem; the REAL problem is that K-complexity is uncomputable, meaning that you cannot in any way prove that the program you’re proposing is, or is NOT, the shortest possible program to express the law.
I disagree; I think the underspecification is a more serious issue than the uncomputability. There are constant factors that outweigh, by a massive margin, all evidence ever collected by our species. Unless there’s a way for us to get our hands on an infinite amount of cputime, there are constant factors that outweigh, by a massive margin, all evidence we will ever have a chance to collect. For any two strings, you can assign a lower complexity to either one by choosing the description language appropriately. Some way to make a good enough (not necessarily optimal) judgement on the language to use is needed for the complexity metric to make any sense.The uncomputability is unfortunate, but hardly fatal. You can just spend some finite effort trying to find the shortest program that produces the each string, using the best heuristics available for this job, and use that as an approximation and upper bound. If you wanted to turn this into a social process, you could reward people for discovering shorter programs than the shortest-currently-known for existing theories (proving that they were simpler than known up to that point), as well as for collecting new evidence to discriminate between them.
I think I’ll give this a try. Let’s start with what a simple non-introspective mind might do:
Init (probably recomputed sometimes, but cached most of the time): I1. Draws a border around itself, separating itself from the “outside world” in its world model. In humans and similarly embodied intelligences you could get away with defining the own body as “inside”, if internal muscle control works completely without inspection.
Whenever deciding on what to output: A1. Generates a list of all possible next actions of itself, as determined in I1. For human-like embodieds, could be a list of available body movements. A2. Computes a probability distribution about the likely future states of the world resulting from each action, consulting the internal world-model for prediction. Resolution, temporal range and world-model building are beyond the scope of this answer. A3. Assigns utilities to each considered future state. A4. Assigns preferences to the probability distribution of futures. This could e.g. use Expected Utility or some satisficing algorithm. A5. Chooses the possible next action with the most-prefered distribution of futures. Tie breaking is implementation defined. A6. Execute that action.
As part of A2, the reactions of other intelligent entites are modeled as part of the general world model. This kind of mind architecture does not model itself in the present; that’d lead to infinite recursion: “I’m thinking about myself thinking about myself thinking about …”. It also wouldn’t achieve anything, since the mind as instantiated necessarily has a higher resolution than any model of itself stored inside itself. It will, however, model past (for lost data) or future versions of itself.
The important point here is that this mind doesn’t model itself while computing the next action. In the extreme case it needn’t have facilities for introspection at all. Humans obviously have some such facilities. Either inbetween deciding on output, inbetween the individual steps A1-A6, completely in parallel, or some combination of those, humans spend cputime to analyze the algorithm they’re executing, to determine systematic errors or possibilities for optimization. I’ll call the neural module / program that does this the introspector. When introspecting a thread which generates motor output (A1-A6 above), one (very effective) assumption of the introspected algorithm will always turn out to be “My own next action is to-be-determined. It’d be ineffective for me to run a model of myself to determine it.”. For a mind that doesn’t intuitively understand the self-reference and infinite-recursion parts, this turns into “My own actions can’t be modeled in advance. I have free will.”.
In the cases where the introspected thread is running another instance of the introspector, the introspector still isn’t attached to its own thread; doing that would lead to infinite recursion. Each introspector will work similarly to the motor-output-choosing algorithm described above, except that the generated output will be in the form of new mental heuristics. Therefore, the same “It’d be ineffective to run a model of myself to determine my next action.” assumption in the algorithm can be observed, and “Free will.” is still the likely conclusion of a mind that doesn’t understand the rationale behind the design.
Unknown wrote: The variation cannot be stopped, since no two physical things will ever be exactly alike.
Assuming we can neglect position in space and movement vectors, different individual elemental particles (e.g. electrons) are already exactly alike. Individual atoms are frequently exactly alike, and the same can be said for molecules. We don’t have a way to precisely copy macroscopic objects yet, but that might change with further technological development. Do you think that Molecular Nanotechnology is fundamentally impossible? If so, why?
And even if you can’t build machines with absolute precision, that doesn’t mean you can’t build them in a way that will prevent them from being actively destructive. If you’re worried that machines you build may become destructive because of construction errors, add a test stage to the construction process, and/or self-test components that will cause the machine to shut down harmlessly when an error is detected. Noticing random corruption isn’t very hard, from a computer science perspective. There are plenty of cheap and effective hash functions available. By adding a bit of redundant data, you can drive the probability of a random corruption being unnotived down to infinitesimal levels.
It’s interesting to note that those oh-so-advanced humans prefer to save children to saving adults, even though there don’t seem to be any limits to natural lifespan anymore.
At our current tech-level this kind of thing can make sense because adults have less lifespan left; but without limits on natural lifespan (or neural degradation because of advanced age) older humans have, on average, had more resources invested into their development—and as such should on average be more knowledgeable, more productive and more interesting people.
It appears to me that the decision to save human children in favor of adults is a result of executing obsolete adaptions as opposed to shutting up and multiplying. I’m surprised nobody seems to have mentioned this yet—am I missing something obvious?
“What men are poets who can speak of Jupiter if he were like a man, but if he is an immense spinning sphere of methane and ammonia must be silent?” Why pick methane and ammonia of all things? Combined, those make up less than 0.4% (original source page) of the planet by volume. Either by volume or by mass, Jupiter mostly consists of molecular hydrogen.
Robert Aumann’s Agreement Theorem shows that honest Bayesians cannot agree to disagree—if they have common knowledge of their probability estimates, they have the same probability estimate.
In addition to what James Annan said, they also both have to know (with very high confidence) that they are in fact honest bayesians. Both sides being honest isn’t enough if either suspects the other of lying.
Minor quibble: since binary 0.1111… is 1, you need a number like 0.1010101… to get an actual counterexample.
Afaict, the original post doesn’t contain any mention of binary fractions. An infinite binary sequence consisting entirely of ones doesn’t represent any finite integer.
Similarly to “Zombies: The Movie”, this was very entertaining, but I don’t think I’ve learned anything new from it.
Z. M. Davis wrote:
Also, even if there are no moral facts, don’t you think the fact that no existing person would prefer a universe filled with paperclips …
Have you performed a comprehensive survey to establish this? Asserting “No existing person” in a civilization of 6.5e9 people amounts to assigning a probability of less than 1.54e-10 that a randomly chosen person would prefer a universe filled with paperclips. This is an extremely strong claim to make!For example, note that the set of people alive includes a significant number of people who are certifiably insane, and in all probability others who, while reasonably sane, have gotten very fed up with various forms of torture inflicted on them over the last few days and might be willing to neglect collateral damage if they could make it stop.
If such a survey were performed, and the results were actually what you claim, I would assign a higher probability to the possibility of a nefarious anti-paperclip conspiracy having infiltrated the survey effort than to the possibility of the results being correct.
Unanimous agreement of our entire species is also a much stronger claim than you need to make for your argument.
For a rather silly reason, I wrote something about:
… explaining the lowest known layer of physics …
Please ignore the “lowest known layer” part. I accidentally committed a mind projection fallacy while writing that comment.
And while a Turing machine has a state register, this can be simulated by just using N lookup tables instead of one lookup table. It seems like we have to believe that 1), the mathematical structure of a UTM relative to a giant lookup table, which is very minimal indeed, is the key element required for consciousness, …
TMs also have the notable ability to not halt for some inputs. And if you wanted to precompute those results, writing NULL values into your GLUT, I’d really like to know where the heck you got your Halting Oracle from. The mathematical structures are very different. For a UTM, the problem of whether it will halt for an arbitrary input is undecidable; in a GLUT with NULL values, you can just look up the input string and are done.
I wonder how this relates to tracking down hard-to-find bugs in computer programs.
And that the tremendous high comes from having hit the problem from every angle you can manage, and having bounced; and then having analyzed the problem again, using every idea you can think of, and all the data you can get your hands on—making progress a little at a time—so that when, finally, you crack through the problem, all the dangling pieces and unresolved questions fall into place at once, like solving a dozen locked-room murder mysteries with a single clue.
This sounds very similar to trying to track down a tricky bug to me. I was going to say that bug-hunting is also almost always original discovery, but the everett-branch/tegmark duplicate argument demolishes that idea. One important difference betwen bug-hunting and scientific discovery is probably the expected effort; even well-hidden bugs usually don’t take months to track down if the programmer focuses on the task.
Imagine a group of 100,000 people, all of whom fit Bill’s description (except for the name, perhaps). If you take the subset of all these persons who play jazz, and the subset of all these persons who play jazz and are accountants, the second subset will always be smaller because it is strictly contained within the first subset.
Nitpicking: Concluding that this is a strict inclusion implicitly assumes that there is at least one jazz player who is not an accountant in the original set. Otherwise, the two subsets may still be equal (and thus, equal in size).
Definitely defect. Cooperation only makes sense in the iterated version of the PD. This isn’t the iterated case, and there’s no prior communication, hence no chance to negotiate for mutual cooperation (though even if there was, meaningful negotiation may well be impossible depending on specific details of the situation). Superrationality be damned, humanity’s choice doesn’t have any causal influence on the paperclip maximizer’s choice. Defection is the right move.
Doug S.:
What character is ◻?
That’s u+25FB (‘WHITE MEDIUM SQUARE’).Eliezer Yudkowsky:
Larry, interpret the smiley face as saying:
PA + (◻C → C) |- I’m still struggling to completely understand this. Are you also changing the meaning of ◻ from ‘derivable from PA’ to ‘derivable from PA + (◻C → C)’? If so, are you additionally changing L to use provability in PA + (◻C → C) instead of provability in PA?
Consciousness might be one of those things that will never be solved (yes, I know that a statement like this is dangerous, but this time there are real reasons to believe this). What real reasons? I don’t see any. I don’t consider “because it seems really mysterious” a real reason; most of the things that seemed really mysterious to some people at some point in history have turned out to be quite solvable.