niplav
I remember Yudkowsky asking for a realistic explanation for why the Empire in Star Wars is stuck in an equilibrium where it builds destroyable gigantic weapons.
Does this include extreme examples, such as pieces of information that permanently damage your mind when exposed to, or antimemes?
Have you made any changes to your personal life because of this?
I predict that this will not become popular, mostly because of the ick-factor around monetary transactions between indviduals that most people have.
However, the inverse strategy seems just as interesting (and more likely to work) to me.
I want to clarify that “AGI go foom!” is not really concerned with the nearness of the advent of AGI, but with whether AGIs have a discontinuity that results in an acceleration of the development of their intelligence over time.
For completion, here’s the prediction on the naive theory, namely that intelligence is instrumentally useful and evolved because solving plans helps you survive:
Instrumental Intelligence Hypothesis: intelligence evolved because it is useful for solving tasks and making plans in nature, and the only reason why humans don't have more of it is because there are metabolic costraints
Isn’t life then a quine running on physics itself as a substrate?
I hadn’t considered thinking of quines as two-place, but that’s obvious in retrospect.
Let the record show that 6 years later, the price of bitcoin has increased 250-fold over the price at the time at which this article was written.
Life is quined matter.
Right, my gripe with the argument is that these first two assumptions are almost always unstated, and most of the time when people use the argument, they “trick” people into agreeing with assumption one.
(for the record, I think the first premise is true)
The child-in-a-pond thought experiment is weird, because people use it in ways it clearly doesn’t work for (especially in arguing for effective altruism).
For example, it observes you would be altruistic in a near situation with the drowning child, and then assumes that you ought to care about people far away as much as people near you. People usually don’t really argue against this second step, but very much could. But the thought experiment makes no justification for that extension of the circle of moral concern, it just assumes it.
Similarly, it says nothing about how effectively you ought to use your resources, only that you probably ought to be more altruistic in a stranger-encompassing way.
But not only does this thought experiment not argue for the things people usually use it for, it’s also not good for arguing that you ought to be more altruistic!
Underlying it is a theme that plays a role in many thought experiments in ethics: they appeal to game-theoretic intuition for useful social strategies, but say nothing of what these strategies are useful for.
Here, if people catch you letting a child drown in a pond while standing idly, you’re probably going to be excluded from many communities or even punished. And this schema occurs very often! Unwilling organ donors, trolley problems, and violinists.
Bottom line: Don’t use the drowning child argument to argue for effective altruism.
I’d be surprised if it could do 5 or 6-digit integer multiplication with >90% accuracy. I expect it to be pretty good at addition.
While this comment might point towards a real phenomenon, it’s phrased in a way I read as passive-aggressive. Tentatively weakly downvoted.
When forecasting, you can be well-calibrated or badly calibrated (well calibrated if e.g. 90% of your 90% forecasts come true). This can be true on smaller ranges: you can be well-calibrated from 50% to 60% if your 50%/51%/52%/…/60% forecasts are each well calibrated.
But, for most forecasters, there must be a resolution at which their forecasts are pretty much randomly calibrated, if this is e.g. at the 10% level, then they are pretty much taking random guesses from the specific 10% interval around their probability (they forecast 20%, but they could forecast 25% or 15% just as well, because they’re just not better calibrated).
I assume there is a name for this concept, and that there’s a way to compute it from a set of forecasts and resolutions, but I haven’t stumbled on it yet. So, what is it?
Open & Welcome Thread – April 2021
After nearly half a year and a lot of procrastination, I fixed (though definitely didn’t finish) my post on range and forecasting accuracy.
Definitely still rough around the edges, I will hopefully fix the charts, re-analyze the data and add a limitations section.
Mostly learning about things, in a “oh, this thing exists! how great!” way. I have detailed some examples (and some failures) here, most notable:
(not in the post) becoming much better at dealing with other people as a result of grokking the typical mind fallacy & how unwell people are most of the time
(not in he post) going to a not-home place to do work and being much more productive (measured & found a relatively strong correlation between going outside the house and productivity)
(not in the post) discovering that I am, in fact, an agent, and can invest money and do something about my appearance and improve it
(not in the post) deciding to stop martial arts and starting to exercise at home as a result of looking at what I value in sport (trying to clearly look at my values and see whether doing X exercise is worth the money/commute).
The Trial by Kafka (intransparent information processing by institutions).
The antimemetics division? Or are you thinking of something different?
Imagine an ultra-intelligent tribe of congenitally blind extraterrestrials. Their ignorance of vision and visual concepts is not explicitly represented in their conceptual scheme. To members of this hypothetical species, visual experiences wouldn’t be information-bearing any more than a chaotic drug-induced eruption of bat-like echolocatory experiences would be information-bearing to us. Such modes of experience have never been recruited to play a sensory or signaling function. At any rate, some time during the history of this imaginary species, one of the tribe discovers a drug that alters his neurochemistry. The drug doesn’t just distort his normal senses and sense of self. It triggers what we would call visual experiences: vivid, chaotic in texture and weirder than anything the drug-taker had ever imagined. What can the drug-intoxicated subject do to communicate his disturbing new categories of experiences to his tribe’s scientific elite? If he simply says that the experiences are “ineffable”, then the sceptics will scorn such mysticism and obscurantism. If he speaks metaphorically, and expresses himself using words from the conceptual scheme grounded in the dominant sensory modality of his species, then he’ll probably babble delirious nonsense. Perhaps he’ll start talking about messages from the gods or whatever. Critically, the drug user lacks the necessary primitive terms to communicate his experiences, let alone a theoretical understanding of what’s happening. Perhaps he can attempt to construct a rudimentary private language. Yet its terms lack public “criteria of use”, so his tribe’s quasi-Wittgensteinian philosophers will invoke the (Anti-)Private Language Argument to explain why it’s meaningless. Understandably, the knowledge elite are unimpressed by the drug-disturbed user’s claims of making a profound discovery. They can exhaustively model the behaviour of the stuff of the physical world with the equations of their scientific theories, and their formal models of mind are computationally adequate. The drug taker sounds psychotic. Yet from our perspective, we can say the alien psychonaut has indeed stumbled on a profound discovery, even though he has scarcely glimpsed its implications: the raw materials of what we would call the visual world in all its glory.
Interview with David Pearce with the H+ magazine, 2009
Or, in other words: I hear your “new colors” and raise you new qualia varieties that are as different from sight and taste as sight and taste are from each other.
Seconding the recommendation. iamef, maybe you want to play around with the dose; the usual dose is too high, and maybe you could take it ~3-4 hours before going to bed. (If you’ve already tried that, please ignore this).