gjm
Karma: 14,075
NewTop
If it took 300k years to develop human software, and 4-13M years to develop human hardware (starting from our common ancestor with chimpanzees), that seems consistent with Eliezer’s claim that developing the software shouldn’t take all that long _compared with the hardware_. (Eliezer doesn’t say “hardware” but “hard-software”, but unless I misunderstand he’s talking about something fairly close to “software that implements what human brain hardware does”.)
[EDITED to add:] On the other hand, you might expect software to evolve faster than hardware, at any given level of underlying complexity/difficulty/depth, because the relevant timescales for selection of memes are shorter than those for genes. So actually I’m not sure how best to translate timelines of human development into predictions for AI development. There’s no very compelling reason to assume that “faster for evolution” and “faster for human R&D” are close to being the same thing, anyway.
Dunno whether it’s an unpopular reaction in any particular circles, but it’s pretty much how I felt about _Neuromancer_ too.
It’s a name for an important special case of “different denominators lead to different averages”, where the cause of the perhaps-unexpected denominator is that some quantities you’re interested in estimating correlate with how likely you are to observe them.
That correlation is a key point here, and any description of the effect that doesn’t include it is describing at most part of it.
That fitted curve looks pretty dubious in its earlier parts. (Maybe I’m misunderstanding and it isn’t a fitted curve at all?)
It looks to me as if 1 and 2 are indeed describing rules of Junto debate, while 3 and 4 (despite the heading that appears in the linked document above the section from which they are taken) are rather describing Franklin’s conduct later in life, inspired by his experiences in the Junto.
[EDITED to add:] In the actual autobiography that heading does not appear; where Franklin says what he did “agreeably to the old laws of our Junto” I don’t think he is claiming that the practice he describes is itself required by those laws; the term “old” is interesting, but I think the Junto was still active at that time—Franklin says it was at about the same time as the establishment of the Philadelphia public library, which was ~1730, and elsewhere in the autobiography he implies the Junto’s continued operation in the late 1730s.
I think it’s possible that you’re interpreting some key words in that provision differently from how Franklin meant them. But I am not very confident either about how he meant them, or about how you are taking them. I think “warmth” may mean “anger” or “heated argument” (and I think you may be taking it to mean “friendliness”); I think “positive” may mean “forceful” (and I think you may be taking it to mean “approving”). The bit about “direct contradiction” seems like evidence for the meanings I am conjecturing; but, again, I am not confident in my guesses about the nuances of 18th-century American English.
I still don’t understand what you’re saying about that first objection. What’s this model in which it “cannot be true” that neither A nor B has higher status than the other?
If you’re saying that that can never happen in a “purely relative” system, then what I don’t understand is why you think that. If you’re saying something else, then what I don’t understand is what other thing you’re saying.
It seems to me that there’s no inconsistency at all between a “purely relative” system and equal or incomparable statuses. Equal status for A and B means that all status effects work the same way for A as for B (and in particular that if there’s some straightforward status-driven competition between A and B then, at least as far as status goes, they come out equal). Incomparable status would probably mean that there are different sorts of status effect, and some of them favour A and some favour B, such that in some situations A wins and in some B wins.
I don’t dispute (indeed, I insist on) the point that it’s vanishingly rare to have no other factors. And I bet you’re right that cleanly separating status effects from other effects is very difficult. It’s not clear to me that this is much of an objection to “purely relative” models of status in contrast to other models. I guess the way in which it might be is: what distinguishes a “purely relative” model is that all you are entitled to say about status is what you can determine from examining who wins in various “status contests”, and since pure status contests are very rare and disentangling the effects in impure status contests is hard you may not be able to tell much about who wins. That’s all true, but I think there are parallel objections to models of “non-relative” type: if it’s hard to tell whether A outranks B because status effects are inseparable from other confounding effects, I think that makes it just as hard to tell (e.g.) what numerical level of status should be assigned to A or to B.
I don’t think I understand your first objection. It seems to say that when there’s a dispute between A and B, and neither A nor B has higher status than the other, onlookers don’t give the benefit of the doubt to either … which is precisely what the relative-status model we’re talking about predicts should happen. How is this an objection?
On the second objection: I agree that many may assume that higher formal-hierarchy position makes a person more likely to be in the wrong. But status is not quite the same thing as position in a formal hierarchy, and I think it’s possible to have both “assume lower-status people have wronged higher-status people rather than the other way around” and “assume formal superiors have wronged formal inferiors rather than the other way around” as heuristics. Also … consider why people might have that latter heuristic. Presumably it’s because higher-ups not infrequently do abuse their authority. Which is to say, they wrong people lower down in the hierarchy and get away with it because of their position.
Of course “no other factors at all” is a vanishingly rare situation. My expectation is that status effects are frequently present but often not alone, and I focused on the situation where there are no other effects for the sake of clarity. When (as usual) there are other effects, the final outcome will result from combining all the effects; the specific effects of status will be hard to disentangle but I see no reason to expect them to vanish just because other things are also present.
First of all, to clarify, questions like those will never be governed purely by considerations of status, and in some cases other factors will matter much more. (Bob might be much higher-status than Carol but I might like Carol much better, or be hoping to persuade her to sleep with me or offer me a job or something.) But to whatever degree those questions’ answers are influenced by status it will be relative “zero-sum” status that matters, because those are relative zero-sum questions.
What I wrote makes it sound like I was suggesting that status is the only, or the dominant, thing determining the answers to those questions. My apologies for writing unclearly. (I think it was just unclarity—I don’t think I thought status was dominant in determining those answers. But it’s easy to forget one’s past states of mind.)
I don’t know whether that suffices to clear things up. In case it doesn’t, some more words about the “Bob and Carol in a fight” scenario: Suppose you see two acquaintances having an argument. Usually this indicates that at least one of them has been unreasonable somehow. Your initial assumption on seeing them cross at one another might be that Bob has been unreasonable to Carol, or that Carol has been unreasonable to Bob. (If you have a sufficiently well trained mind, you may be able to avoid such assumptions. I think many people can’t.) In the—admittedly unlikely—event that there are no other factors at all to favour one of those assumptions over the other, I am guessing (and it is only a guess) that on balance the higher-status person would tend to get the benefit of the doubt, and more people would jump to “Low-Status-Guy has done something stupid / creepy / offensive” than to the equivalent guess about High-Status-Guy.
I agree that celebrity endorsements are probably too complicated to be useful here. I picked on them because they initially seemed like they might be a nice example of non-relative status effects, but the more I thought about it the less convinced I was of that.
Aside from what I take to be a slip-up (your postulate 3 should have {Alice, Bob, Carol} where it currently has {Alice, Carol, Dave}: yes, you’ve correctly described the purely-relative picture of status that I think Bucky had in mind.
I think there is some non-tautologous content associated with this sort of model—namely, the claim that actual status relations are well modelled by this sort of system. That’ll be true in so far as status governs the answers to questions like “if I have to pick one person to do a favour to, which will it be?” or “if Bob and Carol are in a fight, which of them do I expect to be in the right before I have any other information?” and false in so far as status governs the answers to questions like “if it’s Dave’s birthday and I’m buying him a present, how much time and money shall I put into it?” or “when Carol makes a statement, how inclined am I to believe it?“.
It feels to me—but I have no reason to take my feelings as authoritative—as if status is mostly relative but some of these “absolute” questions are influenced by status too. Think about celebrity endorsements for products (of the kind where the celebrity isn’t famous for being expert in a relevant domain): their point is that when people see a very high-status person using a particular brand of $thing and saying “$thing is great!” they’re more inclined to use $thing themselves, and I don’t think it’s plausible that this is driven by some kind of comparison of $famous_person and all the other people in the world who might not be endorsing $thing.
… But maybe this is still relative; maybe the influence I gain, if I become famous, over what brand of shirt or phone people buy, comes at the cost of other famous people’s influence. This would be true e.g. if everyone allocated a roughly fixed amount of attention to seeing what high-status other people are doing so as to copy it, so that when I become famous I’m competing for that attention with Bill Gates and Kanye West, and they get just a bit less of it. So maybe celebrity endorsements aren’t a good example of non-relative status effects after all.
There are a number of Kalman-like things you can do when your updates are nonlinear.
The “extended Kalman filter” uses a local linear approximation to the update. There are higher-order versions. The EKF unsurprisingly tends to do badly when the update is substantially nonlinear. The “unscented Kalman filter” uses (kinda) a finite-difference approximation instead of the derivative, deliberately taking points that aren’t super-close together to get an approximation that’s meaningful on the scale of your actual uncertainty. Going further in that direction you get “particle filters” which represent your uncertainty not as a Gaussian but by a big pile of samples from its distribution. (There’s a ton of lore on all this stuff. I am in no way an expert on it.)
I’m not sure you have shown what you say you have. It may depend on what is meant by “rising in rank” and “dropping in rank”.
The following is obviously true: If people have status values somehow associated with them, these values lying in some sort of partially ordered set, then it is possible for one person’s status value to rise without anyone else’s falling.
I take it, though, that this is not how Bucky sees status as working. The picture is this: there are no separate status values, there are only comparisons—the thing you write as “Rank(A) < Rank(B)”, etc. Now, what does it mean for someone’s rank to rise or fall in this picture? So far as I can see, it has to be something like this: your rank rises if (1) at least one rank-comparison between you and someone else changes in your favour and (2) no rank-comparison between you and anyone else gets worse for you. Likewise (with obvious changes of sign) for “falling”.
If so, then by definition if a change happens that affects only relative rankings involving you, and if that change makes you rise in rank, then it makes at least one other person fall in rank. (Because some rank-comparison changes in your favour, which necessarily means it changes in the other party’s disfavour; and rankings not involving you don’t change, so the other person doesn’t get any compensating improvements vis-a-vis other people.)
The possibility that some pairs of people might be equivalent or incomparable doesn’t change this, so far as I can see, because if the result of comparing my rank with yours improves for me then it necessarily worsens for you—even if one of the “before” or “after” rankings might have been “equivalent” or “incomparable”.
(Arguably, the model of status that I’m foisting on Bucky—Bucky, please let me know and accept my apologies if I’ve guessed wrong—presupposes that status is a zero-sum game. But it might be just as reasonable to say that it is based on the observation that status, at least within a single group, is a zero-sum game. Perhaps that presupposition, or alleged observation, is in fact wrong; but if so, I think actual empirical evidence is what’s needed to show that it’s wrong, rather than theoretical observations about the nature of partial orders.)
It seems like almost everything you’ve described has an alternative description that isn’t about conformity-signalling: employers want employees who are willing to do a lot of work, and if right from the outset you say “I would like to work fewer hours than normal” you are identifying yourself as someone who may not be willing to do a lot of work. So it’s not signalling conformity so much as it’s signalling buy-in to the company’s goals, or submissiveness, or workaholism, or something like that—and obviously employees who are eager to do a lot of work are more valuable (all else being equal) than employees who aren’t. (Which is exactly why that little “work to live, not live to work” speech won’t win their hearts.)
The section on retirement savings spends a lot of space criticizing what people actually do, and rather little saying what they should do instead. Clearly you’re suggesting playing-it-less-safe in some sense, but the ploddingly straightforward “keep investing in hopefully-high-return assets like stocks rather than hopefully-safe assets like bonds, even when growing old” seems (1) not like advice that’s actually justified by anything else you’ve written and (2) to be missing a bigger point (which I take to be related to Ben’s in “Against the barbell strategy”).
It feels like this would benefit from being more explicit. But I get the impression (as I did from Ben’s article) that you are deliberately avoiding being clear and explicit about something here, and I wonder what. (Perhaps nothing! My impression could be wrong.)
Expected log utility? Kelly says to maximize expected log money. (The fact that wealth-derived utility seems to go roughly like log of wealth is largely coincidence, so far as I can see, though I wonder vaguely whether, precisely because of Kelly, brains that value things logarithmically tend to accumulate most in the long run, in which case the Kelly criterion might actually explain why utility is roughly logarithmic in wealth. I’m pretty sure I don’t actually believe that, though.)
Maximizing expected utility also gives the right answer in “win or die”, of course.
Pretty sure it was Hoftstadter (in his Metamagical Themas column; note that the title is an anagram of Gardner’s Mathematical Games) rather than Gardner. You can find the relevant columns, and some further discussion, in his book of that title.
It doesn’t seriously bother me, because I use plain ol’ Less Wrong rather than GW :-). But when a new update drops I like to see how GW is getting on.
The italic variant of the font used for body text in the “Less” theme—I think it’s Source Serif Pro—has a very strange behaviour for me, at the moment, on my laptop running Windows 10. Here is a screengrab at 170% size in Firefox. Look at the serifs and at lowercase “t” and “f”! (And, I guess, everywhere where two strokes somehow run into or across one another.)
If I crank the zoom up one notch more to 200% then the weirdness goes away. In Chrome, it’s weird up to 150% size but normal from 175% up. (My laptop has a “4k” screen, with Windows’ global scaling set to 200%.)
This seems like it must be a text-rendering bug, but I’ve never seen it happen before; I guess something in the Source Serif Pro font is triggering it.
Anyone who makes moral judgements has a Real Moral Something.
But suppose there’s no human-manageable way of predicting your judgements; nothing any simpler or more efficient than presenting them to your brain and seeing what it does. You might not want to call that a system.
And suppose that for some questions, you don’t have an immediate answer, and what answer you end up with depends on irrelevant-seeming details: if we were somehow able to rerun your experience from now to when we ask you the question and you decide on an answer, we would get different answers on different reruns. (This might be difficult to discover, of course.) In that case, you might not want to say that you have a real opinion on those questions, even though it’s possible to induce you to state one.
If it’s a computed trend-line rather than something someone eyeballed then in my book that is a fitted curve. Anyway, that makes sense; presumably it goes below zero somewhere a little to the left of where it stops. Given the obvious discontinuity, it might have made more sense to plot separate lines for before and after...